Source and diversity rate equal to a first transmission rate

ABSTRACT

Packets of real-time information are sent with a source rate greater than zero kilobits per second, and a time or path or combined time/path diversity rate initially being zero kilobits per second. This results in a quality of service QoS, optionally measured at the sender or the receiver. When the QoS is on an unacceptable side of a threshold of acceptability, the sender sends diversity packets at an increased rate. Increasing the diversity rate while either reducing or maintaining the overall transmission rate is new. CELP-based multiple-description data partitioning sends the base or important information plus a subset of fixed excitation in one packet and sends the base or important information plus the complementary subset of fixed excitation in another packet. Reconstruction produces acceptable quality when only one of the two packets is received and better quality when both packets are received. Reconstruction provides for single and multiple lost packets.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a divisional of prior application Ser. No.12/555,485, filed Sep. 8, 2009, now U.S. Pat. No. 8,456,991, issued Jun.4, 2013;

Which was a Divisional of prior application Ser. No. 10/816,351, filedMar. 31, 2004, now U.S. Pat. No. 7,606,164, issued Oct. 20, 2009;

which was a divisional of prior application Ser. No. 09/461,155, filedDec. 14, 1999, now U.S. Pat. No. 6,757,256, granted Jun. 29, 2004;

which claims priority from Provisional Application No. 60/148,191, filedAug. 10, 1999.

The following coassigned patents are hereby incorporated herein byreference:

U.S. Pat. No. 6,496,477, issued Dec. 17, 2002

U.S. Pat. No. 6,421,527, issued Jul. 16, 2002

U.S. Pat. No. 5,987,590, issued Nov. 16, 1999

U.S. Pat. No. 6,179,489, issued Jan. 30, 2001

The following patents relate to this application: U.S. Pat. No.6,757,256, issued Jun. 29, 2004; U.S. Pat. No. 6,804,244, issued Oct.12, 2004; U.S. Pat. No. 6,678,267, issued Jan. 13, 2004; U.S. Pat. No.6,574,213, issued Jun. 3, 2003; U.S. Pat. No. 6,765,904, issued Jul. 20,2004; U.S. Pat. No. 6,801,499, issued Oct. 5, 2004; and U.S. Pat. No.6,801,532, issued Oct. 5, 2004.

FIELD OF THE INVENTION

The present invention relates to the fields of integrated circuits,networking, systems and processes for packet communications, andespecially communication of real time information such as voice, audio,images, video and other real time information over packet.

BACKGROUND OF THE INVENTION

The Internet has long been usable for Internet file transfers and e-mailby packet switched communication. A different technology called circuitswitched communication is used in the PSTN (public switched telephonenetwork) wherein a circuit is dedicated to each phone call regardless ofwhether the circuit is being communicated over in silent periods. Packetswitched networks do not dedicate a channel, thereby sharing a pipe orchannel among many communications and their users. Packets may vary intheir length, and have a header for source information, destinationinformation, number of bits in the packet, how many items, priorityinformation, and security information.

A packet of data often traverses several nodes as it goes across thenetwork in “hops.” In a stream of data, the packets representativethereof may, and often do, take different paths through the network toget the destination. The packets arrive out of order sometimes. Thepackets are not only merely delayed relative to the source, but alsohave delay jitter. Delay jitter is variability in packet delay, orvariation in timing of packets relative to each other due to bufferingwithin nodes in the same routing path, and differing delays and/ornumbers of hops in different routing paths. Packets may even be actuallylost and never reach their destination. Delay jitter is apacket-to-packet concept for the present purposes, and jitter of bitswithin a given packet is a less emphasized subject herein.

Voice over Packet (VOP) and Voice over Internet Protocol (VoIP) aresensitive to delay jitter to an extent qualitatively more important thanfor text data files for example. Delay jitter produces interruptions,clicks, pops, hisses and blurring of the sound and/or images asperceived by the user, unless the delay jitter problem can beameliorated or obviated. Packets that are not literally lost, but aresubstantially delayed when received, may have to be discarded at thedestination nonetheless because they have lost their usefulness at thereceiving end. Thus, packets that are discarded, as well as those thatare literally lost, are all called “lost packets” herein except where amore specific distinction is made explicit or is plain from the context.

The user can rarely tolerate as much as half a second (500 milliseconds)of delay, and even then may avoid using VOP if its quality isperceptibly inferior to other readily available and albeit moreexpensive transmission alternatives. Such avoidance may occur withdelays of 250 milliseconds or even less, while Internet phone technologyhitherto may have suffered from end-to-end delays of as much as 600milliseconds or more.

Hitherto, one approach has stored the arriving packets in a buffer, butif the buffer is too short, packets are lost. If the buffer is too long,it contributes to delay.

If the network is very congested, and the packet is routed by a largenumber of hops, the ratio of lost packets to sent packets in a giventime window interval can rise not just to 5-10% but even to 25% or more,and the real-time communication becomes degraded. VOP quality requireslow lost packet ratio measured in a relatively short time windowinterval (length of oral utterance for instance, with each packetrepresenting a compressed few centiseconds of speech). By contrast, textfile reception can reorder packets during a relatively much longerwindow interval of reception of text and readying it for printing,viewing, editing, or other use. Voice can be multiplexed along withother data on a packet network inexpensively over long distances andinternationally, at low expense compared with circuit-switched PSTNcharges.

A Transport Control Protocol (TCP) sometimes used in connection with theIP (Internet Protocol) can provide for packet tags, detection of lostand out-of-order packets by examination of the packet tags andretransmission of the lost packets from the source. TCP is useful formaintaining transmission quality of e-mail and other non-real-time data.However, the delay inherent in the request-for-retransmission processcurrently may reduce the usefulness of TCP and other ARQ (automaticretransmission request) approaches as a means of enhancing VOPcommunications.

RTP (Real Time Transport Protocol) and RTCP (RTP Control Protocol) addtime stamps and sequence numbers to the packets, augmenting theoperations of the network protocol such as IP. However, these do notprovide QoS (Quality of Service) control.

For real-time communication some solution to the problem of packet lossis imperative, and the packet loss problem is exacerbated inheavily-loaded packet networks. Also, even a lightly-loaded packetnetwork with a packet loss ratio of 0.1% perhaps, still requires somemechanism to deal with the circumstances of lost packets.

A conventional speech compression algorithm has a portion that samples,digitizes and buffers speech in a frame buffer in frame intervals (e.g.20 milliseconds), or frames, and another portion that compresses thesampled digitized speech from one of the frames while more speech isbeing added to the buffer. If the speech is sampled at 8 kiloHertz, theneach 20 millisecond example frame has 160 analog speech samples (8×20).If an 8-bit analog to digital converter (ADC) is used, then 1280 bits(160×8) result as the digitized form of the sampled speech in that 20millisecond frame. Next the compression algorithm converts the 1280 bitsto fewer bits carrying the same or almost the same speech information.Suppose the algorithm provides 8:1 compression. Then 1280/8 bits, or 160bits of compressed or coded speech result from compression. Thecompressed speech is then put in the format of a packet, thus calledpacketized, by a packetizer process.

For every frame of compressed speech in a packet, loss of that packetmeans loss of each frame in that packet. There then arises the problemhow to create 160 bits or more of lost compressed speech. One knownapproach simply repeats the most recent previous frame that is availableat the receiving destination. Another known approach fills the outputframe with silence (zeroes). Reduction of packet loss and packet losshandling strategy are very important challenges in advancing VOPtechnology.

SUMMARY OF THE INVENTION

In one form of the invention, a process of sending packets of real-timeinformation at a sender includes steps of initially generating at thesender the packets of real-time information with a source rate greaterthan zero kilobits per second, and a time or path or combined time/pathdiversity rate, the amount of diversity initially being at least zerokilobits per second. The process sends the packets, thereby resulting ina quality of service QoS, and optionally obtains at the sender a measureof the QoS. Another step compares the QoS with a threshold ofacceptability, and when the QoS is on an unacceptable side of saidthreshold increases the diversity rate and sends not only additionalones of the packets of real-time information but also sends diversitypackets at the diversity rate as increased. Also, rate/diversityadaptation decision may be performed at receiver.

Increasing the diversity rate while either reducing or keeping unchangedthe overall transmission rate is an important new improvement in evensolely-time-diversity embodiments.

Further forms of the invention involve new criteria for initiatingadaptation transitions, and new types of transitions including number ofpackets-per-second transitions, diversity transitions, source ratetransitions and mixtures thereof.

In another form of the invention a single-chip integrated circuitincludes a processor circuit, and a source rate/diversity control. Hereagain, the diversity is contemplated to be time diversity, pathdiversity and combined time/path diversity in various embodiments.

Other forms of the invention encompass other processes, improved packetsand packet ensembles, integrated circuits, chipsets, computer add-incards, information storage articles, systems, computers, gateways,routers, cellular telephone handsets, wireless base stations,appliances, and packet networks, and other forms as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a state transition diagram for a process embodiment ofadaptive control of combinations called states, of source rate anddiversity rate in a media over packet sending computer;

FIG. 2 is a diagrammatic representation of packets in different statesof FIG. 1, wherein time extends horizontally as successive columns inFIG. 2, and the different states correspond to different rows ofdifferently labeled packets in FIG. 2 wherein overall transmission rateis kept limited to less than or equal to that of an s11 state;

FIG. 3 is a block diagram of a system embodiment of a sender computer, anetwork cloud, and a receiver computer showing improvements forrate/diversity adaptation;

FIG. 4 is a family of curves of packet loss rate in percent versusnumber of users N, each curve having a different source rate in kilobitsper second;

FIG. 5 is graph of residual packet loss rate in percent versus speechactivity for two curves in a media-specific redundancy example, a firstcurve corresponding to a source rate and no diversity, and a secondcurve having a lower source rate and with diversity introduced;

FIG. 6 is a diagrammatic representation of packets in 5 transmissionprocesses, wherein time extends horizontally as successive columns inFIG. 6, and the different transmission processes correspond to fivedifferent rows of differently labeled packets in FIG. 6;

FIG. 7 is a family of curves of residual packet loss rate in percentversus speech activity for four curves in a multiple descriptionexample, two of the curves corresponding to a source rate and nodiversity, and two more curves having a respectively lower source rateand with diversity introduced;

FIG. 8 is a family of curves in a multiple description example ofresidual packet loss rate in percent versus number of users N, eachcurve having a different source rate in kilobits per second, and two ofthe curves having diversity as well;

FIG. 9 is a diagram of a RTP packet;

FIG. 10 is another state transition diagram for a process embodimentwith a media-specific redundancy example of adaptive control ofcombinations called states, of source rate and diversity in a media overpacket sending computer;

FIG. 11 is a diagrammatic representation of packets in different states,wherein time extends horizontally as successive columns in FIG. 11, andthe different states correspond to different rows of differently labeledpackets in FIG. 11 wherein overall transmission rate is allowed toexceed that of an s11 state;

FIG. 12 is a block diagram of a simulated network, called a singlebottleneck link simulation having voice sources each described by astate transition diagram inset depicting a two-state Markov voicesource;

FIG. 13 is a graph of simulated network usage by number of users Nversus time t, which is input to the FIG. 12 bottleneck link simulation;

FIG. 14 is a graph of overall transmission rate showing various statesof FIG. 1, versus time, which states are output from the FIG. 12bottleneck link simulation;

FIG. 15 is a block diagram of a combined sending/receiving process,integrated circuit device and system embodiment with adaptiverate/diversity improvements;

FIG. 16 is a flow diagram of a process embodiment of rate/diversityadaptation;

FIG. 17 is partially pictorial, partially block, diagram of integratedcircuits and subsystems for gateways, private branch exchange (PBX)units, wireless base stations, and routers in various embodiments;

FIG. 18 is a block diagram of an improved software system having theimproved integrated circuit device of FIG. 15 as a digital signalprocessor DSP;

FIG. 19 is a partially pictorial, partially block, network diagram withedge devices improved as described herein, for analysis of differentpaths having different selections of improved and unimproved devices atdifferent sites along each of the different paths;

FIG. 20 is a diagram of an RTCP packet for QoS-related reporting from areceiver computer back to a sender computer;

FIG. 21 is a timing diagram of time from left-to-right for sending twoRTCP packets, packet I and packet I+1, and a time interval for a QoScomputation process;

FIG. 22 is a state transition diagram for a process embodiment ofadaptive control of combinations called states, of source rate anddiversity in a media over packet sending computer, wherein criteria formaking various transitions indicated by arrows in FIG. 22 are differentfrom the criteria for making various transitions indicated by the arrowsin FIG. 1;

FIG. 23 is a state transition diagram for a process embodiment ofadaptive control of combinations called states, of source rate anddiversity in a media over packet sending computer, wherein criteria formaking various transitions indicated by arrows in FIG. 23 are differentfrom the criteria for making various transitions indicated by the arrowsin FIG. 1, and suitably supplement the process of FIG. 1;

FIG. 24 is a flow diagram of a process embodiment of rate/diversityadaptation for multicasting, conferencing or other multiple destinationservices which uses part of the process of FIG. 16;

FIG. 25 is a flow diagram of further substeps detailing each of steps1621, 1623 and 1629 of FIG. 16;

FIG. 26 is a flow diagram of further substeps detailing step 1631 ofFIG. 16;

FIG. 27 is a block diagram of an embodiment combining adaptive multipathrouting with adaptive rate/diversity processes, in integrated circuits,devices, computers, systems and networks;

FIG. 28 is a block diagram of software for implementing a networkingprotocol stack useful with the software and system of FIG. 18;

FIG. 29 is a state transition diagram for a process embodiment ofadaptive control of combinations called states, of source rate and firstand second diversity rates in a media over packet computer;

FIG. 30 is a histogram of frequency of consecutive packet losses versusnumber of consecutive packet losses;

FIG. 31 is a state transition diagram for a process embodiment ofadaptive control of combinations called states, of source rate and firstand second diversity rate controlled according to the histogram of FIG.30 in a media over packet computer;

FIG. 32 is a diagrammatic representation of packets in three states ofdiffering numbers of frames per packet combined with a processembodiment of adaptive control of those states in a media-over-packetcomputer; and

FIG. 33 is a state transition diagram for a process embodiment ofadaptive control of combinations called states, of differing numbers offrames per packet and of source rate and diversity rate, in amedia-over-packet computer.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

Various embodiments provide adaptive, robust VoIP/VOP/media over packet(including real time signals over packet) solutions. They provideapproaches to packet network improvements for incorporation intoVoIP/VOP/media-over-packet IETF, TIPHON, and ITU standards. Packet lossresilience encoding and packet loss handling are improved. Adaptivedelay and delay jitter handling contribute to efficient playout andcongestion detection. An adaptive delay and/or delay jitter handlingmechanism is integrated with speech, audio, video and image coders.Constrained rate/diversity adaptation processes and systems embodimentscontrol congestion robustly.

In packet loss resilience encoding and packet loss handling, senderbased diversity embodiments improve G.729 and Texas Instruments codeexcited linear prediction (TI-CELP) codec among other coders. Thefollowing document is hereby incorporated herein by reference for usewhere G.729 is referred to herein: International Telecommunication UnionITU-T G.729 (03/96) Telecommunication Standardization Sector of ITU,General Aspects of Digital Transmission Systems, Coding of Speech at 8kbit/s Using Conjugate-Structure Algebraic-Code-ExcitedLinear-Prediction (CS-ACELP), ITU-T Recommendation G.729.

For example, information about packet n is sent in packets {n+k: k>0} ina packet sequence:[P(n−1)′P(n)][P(n)′P(n+1)][P(n+1)′P(n+2)][P(n+2)′P(n+3)]

Computationally-efficient CELP based important information redundancyschemes are provided.

Computationally-efficient multiple description CELP coding is provided.

Adaptive delay and adaptive delay jitter handling advantageouslycompensate delay variation in arriving packets, detect delay-spikes ofdelay value due to congestion, and increase playout delay and sendcongestion notification.

Adaptive delay and adaptive delay jitter handling process is suitablyintegrated with G.729 codec and other codecs.

Combined adaptation on both source rate sij and packet network diversityrate dij, a process called rate/diversity adaptation herein, robustlycontrols congestion. Some embodiments herein use source rate adaptationalone, with advantageous simplicity and QoS improvement compared toapproaches hitherto. Also, further embodiments use diversity adaptationalone or combined with source rate adaptation with the followingadvantages for real-time traffic:

-   -   overcome distributed congestion    -   handle heterogeneous traffics    -   overcome packet losses due to bit errors (as in modem/satellite        links)    -   compensate for packet losses due to processing limitations,        late-arrival, etc.    -   recognize that during DTX (discontinuous transmission), feedback        is not provided.

To handle congestion, TCP reduces the number of packets transmitted anduses retransmission which often introduces unacceptable delay and delayjitter for real-time traffic. In rate/diversity adaptation for real-timecommunication, diversity is advantageously introduced.

As is described herein, rate/diversity adaptation for robust congestioncontrol offers features in one or another of the embodiments, such as

-   -   Overall transmission rate is reduced during congestion    -   Increases source rate through multiple stages    -   Avoids oscillations between states by incremental changes based        on different thresholds or other transition criteria    -   Robust adaptation mechanism or process    -   Adaptation mechanism takes into account loss, high delay and        delay jitter    -   Combines adaptive delay/delay-jitter handling, for congestion        detection and playout adaptation    -   Works with multiple description (MD) packets improved with        diversity processes, devices and systems    -   Works with Important Information (I-I) based redundancy packets        improved with diversity processes, devices and systems    -   Improves other redundancy packet networking techniques both with        new temporal diversity and path diversity processes, devices and        systems.

Important areas of improvement for VoIP/VOP technology involveminimizing delays inside computers and their software, lowering networklatency, and tightening network jitter. One or more of these advantagesare conferred by some of the embodiments described herein.

By adapting transmission rate and the amount of time or path or combinedtime/path diversity in VoIP/VOP applications, robust solutionsadvantageously handle network impairments and congestion, whileutilizing network resources efficiently.

Improvements in VoIP/VOP processes, integrated circuits and systemsutilizing path diversity are described in the coassigned patent U.S.Pat. No. 6,496,477 “Integrated Circuits, Systems, Apparatus, Packets andProcesses Utilizing Path Diversity for Media Over Packet Applications,”which is incorporated herein by reference. In one category ofembodiments, the skilled worker uses the circuits and methods describedin the incorporated material and adds the adaptive features furtherdescribed herein.

RTP/UDP/IP protocols do not offer QoS control mechanisms. Hence, VoIPapplications, if they were to use RTP/UDP/IP protocols, suffer fromfluctuations in network conditions and poor voice quality can result.One approach for QoS control involves source rate control, with nodiversity, wherein one approach for QoS control is to adapt the sourcerate to the fluctuations in network conditions, per “Reducing bandwidthrequirements,” Micom Whitepaper, 1998; D. Sisalem et al., “Theloss-delay based adjustment algorithm: A TCP-friendly adaptationscheme,” NOSSDAV, (International Workshop on Network and OperatingSystem Support for Digital Audio and Video), July 1998. However, thisapproach may not handle short-term network fluctuations well, and iscomplicated as VoIP/VOP applications often involve multiple links ofheterogeneous characteristics. First, there is a need to locate the“bottleneck” link, and, second, all users of the bottleneck link may notreduce their transmission rate.

In time diversity, information about packet n is also transmitted inpacket n+1 and sometimes in even further packets where packets having atleast some information in common with each other are called dependentpackets.

Path Diversity sends dependent packets over two or more paths in thenetwork, thus increasing the probability of recovering the informationthat was coded to produce the dependent packets.

Combined Time/Path Diversity approach uses both processes of TimeDiversity and Path Diversity in innovative ways.

“Diversity packet,” where the term is used herein sometimes means aself-contained packet with its own header and diversity information.However, the term “diversity packet” can also mean diversity bits andextra header bits put in a packet that already has a header and apayload.

Time diversity schemes provide inter-packet diversity, by includinginformation about the nth packet in succeeding packets {n+k: k>=1}. Theymay employ redundancy schemes (media-specific redundancy, forward-errorcorrection FEC) and multiple-description schemes, for instance.

In the example sequence of four packets just below, bits P(n) representprimary packets, and P(n)′ and P(n)′ each represent instances ofdiversity. This packet sequence has a number of diversity stages (here 3namely P(n), P(n)′ and P(n)″) and a diversity length of 4. Diversitylength is the minimum number of packets in a symbol sequence needed todefine the diversity used.P(n)P(n−1)′P(n−3)″P(n+1)P(n)′P(n−2)″P(n+2)P(n+1)′P(n−1)″P(n+3)P(n+2)′P(n)″

The diversity length is greater than the number of diversity stagesbecause of diversity offset, which is one here. Diversity offsetcorresponds in this case to absence of P(n) or any primed P(n) in thethird packet.

Redundancy schemes piggyback a version/function (media-specificredundancy/FEC) of nth packet to (n+k)th packet, k>=1, as shownhereinbelow. The following sequences of packets are examples ofmedia-specific redundancy schemes:

P0 P1 P0′ P2 P1′ P3 P2′ . . . P0 P1 P2 P0′ P3 P1′ P4 P2′ . . . P0 P1 P0′P2 P1′ P0′ P3 P2′ P1′ P4 P3′ P2′ . . . P0 P1 P0′ P2 P1′ P0′′ P3 P2′ P1′′P4 P3′ P2′′ . . .

where Pn denotes nth packet, and Pn′ and Pn″ denote versions of the nthpacket. “Version” here means a dependent datum in a packet having atleast some information included or encoded therein which corresponds toanother packet. Thus, the juxtaposition of symbols used above signifiesconcatenation in the description of a given packet, and any possiblesequence of concatenation in every packet is contemplated, as well asthe order of concatenation literally shown for each packet.

Multiple-description (MD) schemes break the input stream into multipledescriptions, for instance, using MD quantizers [V. A. Vaishampayan etal., “Asymptotic analysis of multiple description quantizers,” IEEETrans. On Inform. Theory, January 1998}. Here none of the descriptionshave the full information intended for reception, and instead each ofthe descriptions has less than that full information, and thedescriptions which are received (even if some be lost) then have theirinformation combined in the decoder to obtain what information isavailable in them collectively. The following packet sequences symbolizeexamples of embodiments of process and systems

P0 P1 P0′ P2 P1′ P3 P2′ . . . P0 P1′ P0′ P2 P1 P3′ P2′. . . P0 P1 P0′P2′ P1′ P3′ P2 P4 P3 P5 P4′ P6′ P5′ P7′ . . . P0 P1 P2 P0′ P3 P1′ P4 P2′. . . P0 P1 P0′ P2 P1′ P0″ P3 P2′ P1″ P4 P3′ P2″ . . . P0 P0′ P1′ P0″P1″ P2″ P1 P2 P3 P2′ P3′ P4′ P3″ P4″ P5″ P4 P5 P6. . .

where Pn, Pn′ and Pn″ denote multiple descriptions of the nth packet.

One type of embodiment uses plural types of time diversity concurrentlyso that Redundancy is applied concurrently with Multiple-Description.

Among other advantageous things herein, the present applicationdescribes path diversity processes, integrated circuits and systemswhereby VoIP/VOP software applications open multiple (two or more) flowsbetween the same source and the destination. The packets in each flowtraverse separate paths from packets in other flows (for at least someof the hops between the source and destination). By having multiplepaths, or causing multiple network paths to be accessed, used andtraversed, such path diversity processes, integrated circuits andsystems reduce, as between the diverse flows, the correlation of packetloss, delay, jitter and other less than desirable metrics of performancewhich are ameliorated herein.

Some examples of combined time/path diversity embodiments for VOP thatconfer advantageously efficient bandwidth utilization are describednext.

1. Combined time/path diversity. The time diversity is redundancy,multiple description, FEC forward error correction, or other suitableprocess.

a) Path 1: P0 P1 P0′, . . . Path 2: P0 P1 P0′, . . . b) Path 1: P0 P1P0′ P2 P1′, . . . Path 2: P0 P1 P0′ P2 P1′, . . .(In embodiment 1(b), the packet stream of Path 2 is time-delayedrelative to the packet stream of Path 1.)

Path switching diversity randomizes bursty packet losses withoutincreasing bandwidth utilization. First create multipleconnections/paths (e.g. 2 connections) between the source and thedestination. Then transmit as follows:VOP packet stream: P0P1P2 . . . P(n−1)P(n)P(n+1)P(n+2) . . .

Path 1: P0 P2 P(2n) Path 2: P1 P3  P(2n + 1)

Generally speaking, a set of n respective packets are directed into acorresponding number n of respective diverse network paths whereupon theprocess repeats for the next set of n packets, and so on.

Combined path-switching diversity/redundancy embodiments combinepath-switching diversity and redundancy processes, and advantageouslyachieve good voice quality with efficient bandwidth utilization. In atwo-path embodiment

Path 1: P0 P2 P1′ P4 P3′ . . . Path 2: P1 P0′ P3 P2′ . . .

In this approach, n respective redundancy packets are directed into acorresponding number n of respective diverse network paths whereupon theprocess repeats for the next set of n packets, and so on.

Combined path-switching diversity/multiple-descriptions embodimentscombine path-switching diversity and multiple-description processes, andadvantageously achieve good voice quality with efficient bandwidthutilization. In a two-path embodiment

Path 1: P0 P2 P1′ P4 P3′ . . . Path 2: P1 P0′ P3 P2′ . . .

In this approach, n respective multiple-descriptions packets aredirected into a corresponding number n of respective diverse networkpaths whereupon the process repeats for the next set of n packets, andso on.

5. In an embodiment of incorporated coassigned patent U.S. Pat. No.6,496,477, the original voice packet stream (P0 P1 P2) is sent in itsentirety, on path 1 and on path 2.

Path 1: P0 P1 P2 P3 P4 . . . Path 2: P0 P1 P2 P3 P4 . . .

The following Performance Table summarizes performance for a two-pathsystem of the above four embodiments. The symbols R, H and R′ refer tosource bit rate, header bit rate, and redundancy bit rate, respectively.Concealment processes such as interpolation are suitably used to recovermissing parts of a media stream. In G.729, a frame erasure concealmentmethod specified in G.729 spec is suitably used when the path diversityfeature herein is not being used.

PERFORMANCE TABLE EMBODIMENT BANDWIDTH QUALITY 1(a), 1(b) 2(R + H + R′)Full quality when any of the paths is good. Acceptable quality when evenboth paths are bad. 2 (R + H) Packet-loss concealment, when any of thepaths is bad. 3 (R + H + R′) Full quality, when both paths are good.Acceptable quality, when any of the paths is good. Packet-lossconcealment, when both paths are bad. 4 (R + H + R′) Full quality, whenboth paths are good. Acceptable quality when any of the paths is good.Packet-loss concealment, when both paths are bad. 5 2(R + H) Fullquality, when both paths are good. Packet-loss concealment, when bothpaths are bad.

In a similar manner, the skilled worker analyzes various embodiments andselects for implementation whichever one(s) are most suitable for theparticular needs at hand.

Media-specific redundancy schemes piggy-back a version of nth packet to(n+k)th packet. In VoIP/VOP heretofore a separate encoding schemegenerated the redundancy version of the nth packet, or piggybacked theentire nth packet to the (n+k)th packet. Hereincomputationally-efficient CELP (code-excited linear prediction) baseddiversity embodiments for VoIP/VOP are described, such as for generatingmedia-specific redundancy information. These are herein called“Important Information” based diversity embodiments. Some ImportantInformation based diversity embodiments use base information, orImportant Information, from CELP encoding as redundancy information, toachieve diversity. Below, two embodiments are described in more detailfor G.729. These embodiments are given for two stages (primary stageplus one secondary stage), with no diversity offset. Embodiments includeextensions based on more than two stages, and diversity offsets.

Embodiment 1

With no pulses in secondary stage. Using G.729, the secondary stage(redundancy stage) has these Important Parameters—LPC (Linear PredictiveCoding) parameters, LTP (Longterm Prediction) lags, parity check, andadaptive and fixed codebook gains—according to the sequenceP(n)P(n−1)′P(n+1)P(n)′P(n+2)P(n+1)′P(n+3)P(n+2)′

1A. Reconstruction with single packet loss is shown in the next sequencebelow. The LPC parameters, LTP lags, parity check, and adaptive andfixed codebook gains are obtained from the secondary stage. Theexcitation reconstruction mechanism is suitably made to be thereplacement excitation generation scheme described in the G.729 standardsection 4.4.4 with the following modification. For lost-framesconsidered as nonperiodic, the adaptive codebook contribution is set tozero only if the absolute value of the adaptive codebook gain (obtainedfrom the secondary stage) is less than 0.4, otherwise the adaptivecodebook contribution is reconstructed from the adaptive codebook gainand LTP lag obtained from the secondary stage.P(n)P(n−1)′[Lost Packet]P(n+2)P(n+1)′P(n+3)P(n+2)′P(n)[P(n+1)′|-](excitation)P(n+2)P(n+3)(where “excitation” shown above refers to reconstruction of the dashedpart of the packet symbols)

1B. Reconstruction with two or more consecutive packet losses is shownin the next sequence below. Now the packet (n+2) is reconstructed asdescribed in the paragraph 1A just above. The packet (n+1) isreconstructed by the G.729 frame erasure concealment scheme specified inthe G.729 standard section 4.4, used for packet loss concealment. Thesteps of section 4.4 are repetition of synthesis filter parameters(4.4.1) attenuation of adaptive and fixed-codebook gains (4.4.2),attenuation of the memory of the gain predictor (4.4.3), and generationof the replacement excitation (4.4.4).P(n)P(n−1)′[Lost Packet][Lost Packet]P(n+3)P(n+2)′P(n)[-][P(n+2)′|-](excitation)P(n+3)

Embodiment 2

With pulses in secondary stage. Using G.729, the secondary stage(redundancy stage) has LPC parameters, LTP lags, parity check, andadaptive and fixed codebook gains, and first few or all fixed codebookpulses.

2A. In reconstruction with single packet loss, the LPC parameters, LTPlags, adaptive and fixed codebook gains, and the included pulses areobtained from the secondary stage. The remaining fixed codebook pulsesare set to zero.

2B. Reconstruction with two or more consecutive packet lossesreconstructs the packet (n+2) as described in the paragraph 2A justabove. The packet (n+1) is reconstructed by the G.729 frame erasureconcealment scheme specified in the G.729 standard section 4.4, used forpacket loss concealment, when there is no diversity.

Multiple-description data partitioning based diversity embodiments aredescribed next.

It is believed that heretofore there has been no CELP-based multipledescription process. Herein are described computationally-efficient,CELP-based multiple description embodiments using multiple-descriptiondata partitioning. Parentheses are used in the next few sentences topoint out certain significant combinations of information.

These embodiments send (the base or important information+a subset offixed excitation) in one packet and (the base or importantinformation+the complementary subset of fixed excitation) in anotherpacket. Below, two embodiments are described in more detail for G.729.These embodiments are given for two stages, with no diversity offset.Embodiments include extensions based on more than two stages and,diversity offsets.

DEFINITION: Multiple description data partitioning: In this approach,(the base information+a subset of enhancement information) is sent inone packet, and (the base information+the complementary subset ofenhancement information) is sent in another packet. Here, when only oneof the packets is received at the receiver, to produce acceptablequality that packet is reconstructed. When both packets are received atthe receiver, they both are combined to produce better quality.

Embodiment 3

with no pulses in the base or important information. Using G.729, thefirst stage has LPC parameters, LTP lags, parity check, adaptive andfixed codebook gains, and every other fixed codebook pulses. The secondstage has LPC parameters, LTP lags, parity check, adaptive and fixedcodebook gains, and the remaining fixed codebook pulses. See sequencebelow:P(n)P(n−1)′P(n+1)P(n)′P(n+2)P(n+1)′P(n+3)P(n+2)′

3A. In reconstruction with single packet loss, for packet n and packet(n+1), only one stage is used for reconstruction, and the remainingfixed codebook pulses are set to zero (note that these pulses includethe fixed codebook pulses from the lost diversity stage). Seereconstruction below:Received: P(n)P(n−1)′[Lost Packet]P(n+2)P(n+1)′P(n+3)P(n+2)′Reconstructed:[P(n)|-](excitation)[P(n+1)′|-](excitation)P(n+2)+P(n+2)′P(n+3)+P(n+3)′(The plus (+) sign refers to combination of information forreconstruction).

3B. Reconstruction with two or more consecutive packet lossesreconstructs the packet n and the packet (n+2) as described in theparagraph 3A just above. The packet (n+1) is reconstructed by the G.729frame erasure concealment scheme specified in the G.729 standard section4.4, used for packet loss concealment. See reconstruction below:P(n)P(n−1)′[Lost Packet][Lost Packet]P(n+3)P(n+2)′[P(n)|-][- - - -][P(n+2)′|-](excitation)P(n+3)+P(n+3)′

Embodiment 4

With pulses in the base or important information. Using G.729, the firststage has LPC parameters, LTP lags, parity check, adaptive and fixedcodebook gains, first few fixed codebook pulses, and every other fixedcodebook pulses from the remaining pulses. The second stage has LPCparameters, LTP lags, parity check, adaptive and fixed codebook gains,the same first few fixed codebook pulses, and the complementary subsetof pulses from the remaining fixed codebook pulses. See sequence below:P(n)P(n−1)′P(n+1)P(n)′P(n+2)P(n+1)′P(n+3)P(n+2)′

4A. In reconstruction with single packet loss, for packet n and packet(n+1), only one stage is used for reconstruction, and the remainingfixed codebook pulses are set to zero. See reconstruction below:P(n)P(n−1)′[Lost Packet]P(n+2)P(n+1)′P(n+3)P(n+2)′[P(n)|-](excitation)[P(n+1)′|-](excitation)P(n+2)+P(n+2)′P(n+3)+P(n+3)′

4B. Reconstruction with two or more consecutive packet lossesreconstructs the packet n and the packet (n+2) as described in theparagraph 4A just above. The packet (n+1) is reconstructed by the G.729frame erasure concealment scheme specified in the G.729 standard section4.4, used for packet loss concealment. See reconstruction below:P(n)P(n−1)′[Lost Packet][Lost Packet]P(n+3)P(n+2)′[P(n)|-][- - - -][P(n+2)′|-](excitation)P(n+3)+P(n+3)′Extensions

Further embodiments are contemplated with

diversity offset,

multiple stages, and

multiple stages and diversity offsets.

Regarding performance and delay due to diversity: If the packet delayvariation is larger than the packet interval/size, the system may choosenot to introduce additional delay while making use of diversity in alimited manner.

Some other embodiments augment the MD (multiple description) approach asfollows. For fixed codebook search, minimize [error(full rate)+w1error(Description 1)+w2 error(Description 2)] instead of minimizingerror(full rate) alone. (The letters “w1” and “w2” symbolize weightcoefficients. Description 1 and Description 2 symbolize twodescriptions). In addition an interpolation filter is used forshaping/filling of excitation. Also, MD quantizers are used for LPCparameters, LTP lags, fixed codebook gain and adaptive codebook gain.

Some Important Information embodiments apply FEC to importantinformation.

Still other embodiments combine interleaving and diversity.

Some diversity based embodiments add interpolation of parameters inaddition to fixed excitation repeating, from available (past/future)frames.

Adaptive Rate/Diversity Processes

In a type of constrained adaptive rate/diversity processes, integratedcircuits and systems herein, these adapt source rate and the amount oftime or path or time/path diversity, in accordance with networkfluctuations based on some QoS level measure (e.g., overall packet lossrate due to packet loss, delay, delay-jitter, etc., but before theapplication or compensation with diversity). Note that QoS is an inversefunction of packet loss rate—in other words, QoS goes up as packet lossrate goes down. Thus, being higher than a threshold of QoS means beingless than a corresponding threshold of packet loss rate. Put yet anotherway, QoS is a positive quantity and packet loss rate can be thought ofas a negative quantity.

Further details of some adaptation process embodiments are as follows:

When QoS level measure is lower than a given QoS threshold (e.g.,overall packet loss rate before the application of diversity exceeds (>)Threshold1), increase/introduce diversity while decreasing overalltransmission rate or keeping overall transmission rate substantiallyunchanged.

B. When QoS level measure is higher than another QoS thresholdrepresenting higher quality of service than the given QoS threshold ofparagraph A (e.g., overall packet-loss rate before the application ofdiversity is less than (<) Threshold2 where Threshold2 is less than orequal (<=) Threshold1), increase source rate (the bit rate for packetstream Pn). Note that Threshold) Th1 and Threshold2 Th2 are values ofthe packet loss rate metric, inversely related to QoS and thresholds ofQoS. The method for determining new steady-state source rate depends onavailable network resources according to any suitable table, algorithmor method selected by the skilled worker, see examples herein. Theprocess of increasing source rate is achieved through one or more stagesor steps. Two different steps which can either be used alone, orconsecutively, or concurrently, are

When increasing the overall transmission rate, maintain diversity.

When reducing the amount of diversity, do not increase the overalltransmission rate.

Note that source rate (sij) is different from “overall transmissionrate.” Overall transmission rate for purposes herein denotes the sumsij+dij in a given state. Roughly speaking, overall transmission rate isthe sum of the packet P0 rate plus packet P0′ rate plus rates for anyother diversity packet for P0-primed.

By the use of diversity, some process embodiments handle short-termnetwork fluctuations well, cope with VoIP/VOP applications that involvemultiple links of heterogeneous characteristics, and are TCP-trafficfriendly. This is because the overall transmission rate is decreased, orat least not increased, on the network in the event of low QoS level,thereby not burdening the network increasingly, as these processembodiments work to ameliorate the QoS. In this way such processembodiments improve QoS for users and are network friendly in that suchprocess embodiments could be implemented at one node, some nodes, or allnodes without further congesting the network while improving QoS.

In one example, packet loss rate Threshold1 is selected to be threepercent (3%), and packet loss rate Threshold 2 is selected to beone-half percent (0.5%). Packet size is forty (40) milliseconds,corresponding to an overhead (header rate) of 320 bits/40 msec=8 kbpsfor VoIP. RTCP Transmission Interval is set to five (5) seconds, and thefraction lost (or packet-loss rate) is computed during last five (5)seconds in a latter part of the RTCP Transmission Interval. (Use of RTPand RTCP is described further later hereinbelow.) Source rate selectionss11, s12, s21, s22, s31, s32 are established at 16.0, 11.2, 11.2, 8.0,8.0 and 5.7 kilobits/sec respectively. Diversity selections d11, d12,d21, d22, d31, d32 are established at 0.0, 4.8, 0.0, 3.2, 0.0 and 2.3kilobits/sec respectively. All the foregoing values are, of course,offered illustratively and not in any limiting sense.

QoS level measure computation is temporally localized at the suggested 5seconds in order to avoid smoothing of network effects. The exampleadaptation mechanism uses a high threshold (Threshold1) when QoS leveldecreases, and uses a low threshold (Threshold2) when QoS levelimproves. This approach advantageously addresses a scenario of possibleoscillation between rate/diversity states, but where this scenario isnot applicable or is addressed by other means such as delay processes orotherwise, then some embodiments can also use equal thresholds or anychoice of thresholds that confers satisfactory adaptation.

Some adaptation embodiments take into account packet loss, high delay,and delay-jitter, such as in the QoS level measure process. In one typeof process embodiment, overall packet-loss rate due to loss, delay anddelay jitter (and before the application of diversity) is used as QoSlevel measure.

Two specific embodiments or realizations of a rate/diversity adaptationprocess or method are diagrammed in the state transition diagrams ofFIGS. 1 and 2. Note that these are not an exhaustive set of embodimentsor realizations. If a connection is bad, one type of embodiment reducesthe source rate and adds diversity while taking the overall transmissionrate, or network burden, into account.

A state transition diagram is well understood by the skilled worker, andgenerally speaking, the arrows are transitions which occur upon theexistence of a condition noted near its respective arrow. Thus, in FIG.1, the arrows join small circles representative of a state of onesending computer connected to a packet network. More particular thestate is the state of a source rate and diversity control block (331 ofFIG. 3 discussed later hereinbelow), which thereby controls the state ofa speech encoder (321 of FIG. 3), audio encoder or other media sourceencoder or compressor.

FIG. 1 illustrates a state transition diagram for a process embodimentwhere a new steady-state source rate is designated s11. In FIG. 1,overall packet loss rate F exceeds (>) Threshold1 before the applicationof diversity. (Where the phrase “packet loss rate” is used herein, it isto be understood interchangeably with packet loss fraction or packetloss ratio which are similar concepts.) When Threshold1 is exceeded bypacket loss rate F, a transition 101 occurs from the state (s11,d11) tostate (s22,d22). When a gatekeeper request GK or Buffer occupancy fullsignal occurs, a transition 102 occurs from the state (s11,d11) to state(s21,d21). Gatekeeper and/or router boxes in the network may signalrequests as just mentioned, and these are included in the statetransition diagram for context and completeness of description.

Note that dotted ovals 111, 113, 115, etc. diagrammatically surround andthus indicate states that have the same sum of s and d components, andthus indicate essentially the same “overall transmission rate” (i.e.,same network burden or load). From left to right in each of FIGS. 1, 10,22, 23, 29 and 31, the ovals indicate progressively reduced overalltransmission rate, or sum of s and d components.

In FIGS. 1 and 2, an optional constraint in one class of embodimentsintroduces a specification: do not increase overall transmission rate.Also,Overall transmission rates: . . .(s11+d11)=(s12+d12)>(s21+d21)=(s22+d22)>(s31+d31)=(s32+d32) . . . .

where (d11<d12), (d21<d22) and (d31<d32).

sij denotes the rate for Pn, and dij denotes the rate for Pn′.

In FIG. 2 the bit length of each sij or dij part of a packet isproportional to the source rate and diversity rate when the transmissionof packets themselves is at a constant rate of issuing packets from thesender.

In FIG. 2 the source rate of a packet stream 205 is a given amount s11.When QoS falls, the control block 331 of FIG. 3 utilizes a processwherein it reduces source rate to an amount s22 and introduces a smalldiversity rate of d22 as shown in packet stream 215. Original overalltransmission rate s11+0=s11 of packet stream 205 exceeds overalltransmission rate s22+d22 of packet stream 215 resulting fromrate/diversity adaptation. The overall transmission rate of packetstream 215 has been lowered or reduced from that of packet stream 205.

Further in FIG. 2 an alternative process goes from source rate s11 ofpacket stream 205 to a packet stream 225. Packet stream 225 has anoverall transmission rate comprised of a source rate s22 and diversityrate d22 that sum to an amount substantially equal to original rate s11.

Transitions like transition 101 from left to right in FIGS. 1 and 10indicate that the process has encountered a QoS degradation where packetloss rate is exceeding Threshold1, and thus has become unacceptable. Inresponse the process is ameliorating the QoS, by lowering the sourcerate and adding diversity until the QoS improves enough that the packetloss rate has gotten below Threshold2 Th2. Lowering source rate meansthat the original information is being coded with more compression andperhaps more lossiness of compression, but this compression loss isalmost insignificant compared to the user-perceived degradation thatpacket loss causes.

Conversely, transitions like 103 and 105 from right to left in FIG. 1indicate that the process is increasing its use of the network at a timewhen QoS has improved sufficiently to permit such increased use. Suchincreased use takes the forms of increasing the source rate and reducingand/or terminating path diversity, time diversity or combined time/pathdiversity.

FIG. 10 illustrates a state transition diagram for a process where a newsteady-state source rate is designated s21. In FIG. 10 the processsuitably is arranged to make transition 1005 inside oval 113 when QoShas improved to the extent that packet-loss rate F has fallen belowThreshold 2. This FIG. 10 example shows that embodiments are alsosuitably arranged to stay in a lower state (s21, d21). Here, as in FIG.1, operations move from a higher source rate s11 to a lower source rates22 if QoS degrades (transition 101).

Thresholds can be varying as well, such as depending on the source rateused. Thus, in FIG. 10, a further transition 1007 occurs when criterionF exceeds a threshold Th3. In one example, transition 101 occurs whenloss fraction exceeds 3% and transition 1007 occurs when loss fractionexceeds 4%.

Further details of some more adaptation process embodiments are asfollows:

1. Adapt both the source rate sij and the amount of diversity dij, inaccordance with network fluctuations based on some QoS level measure(see examples in FIGS. 2 and 11).

When QoS level measure is lower than a given QoS threshold (e.g.,overall packet loss rate before the application of diversity exceeds (>)Threshold1), add diversity while decreasing the source rate or keepingthe source rate unchanged. (i.e., source rate sij here instead ofoverall transmission rate sij+dij in some embodiments)

-   -   b) When QoS level measure is higher than another QoS threshold        representing higher quality of service than the given QoS        threshold of paragraph (a) (e.g., overall packet-loss rate        before the application of diversity is less than (<) Threshold2        where Threshold2 is less than or equal (<=) Threshold1),        increase source rate. Note that Threshold1 Th1 and Threshold2        Th2 are values of the packet loss rate metric, inversely related        to QoS and thresholds of QoS. The method for determining new        steady-state source rate depends on available network resources        according to any suitable table, algorithm or method selected by        the skilled worker, see examples herein. The process of        increasing source rate is achieved through one or more stages to        reduce or prevent oscillatory behavior.

2. Special case: Adapt both the source rate and the amount of diversity,in accordance with network fluctuations based on some QoS level measureand constrain their sum so that overall transmission rate is reduced orunchanged (see FIG. 2).

When QoS level measure is lower than a given QoS threshold (e.g.,overall packet loss rate, packet loss rate before compensating bydiversity, exceeds (>) Threshold1), add diversity while decreasing theoverall transmission rate or keeping the overall transmission rateunchanged.

-   -   b) When QoS level measure is higher than another QoS threshold        representing higher quality of service than the given QoS        threshold of paragraph (a) (e.g., overall packet-loss rate,        packet loss rate before compensating by diversity, is less than        (<) Threshold2 where Threshold2 is less than or equal (<=)        Threshold1), increase source rate. Note that Threshold1 Th1 and        Threshold2 Th2 are values of the packet loss rate metric,        inversely related to QoS and thresholds of QoS. The method for        determining new steady-state source rate depends on available        network resources according to any suitable table, algorithm or        method selected by the skilled worker, see examples herein. The        process of increasing source rate is achieved through one or        more stages.

3. As noted in 1 and 2, the process of increasing source rate isachieved through one or more stages. Two different steps which caneither be used alone, or consecutively, or concurrently, are a) whenincreasing the overall transmission rate, maintain some diversity, andb) when reducing the amount of diversity, do not increase the overalltransmission rate. Note that (a) and (b) can be realized using variouscombinations of source rate and diversity.

4. The approaches are applied to a) time diversity embodiments(media-specific redundancy, important information diversity, FEC,multiple description, multiple description data partitioning), b) pathdiversity embodiments, and c) combined time diversity and path diversityembodiments.

5. QoS level measure computations and Adaptation Logics.

5A. Delay jitter handling via fixed-delay threshold embodiment declaresa packet as lost, if the end to end delay of the packet is greater thana fixed threshold. Overall packet loss rate due to loss, delay, anddelay jitter (but before the application of diversity) is used as a QoSlevel measure. Thus, in FIG. 1, transitions 101 and 103, 105 occur onthresholds as shown compared to a value F which is overall packet-lossrate in this 5A embodiment. Preferably but not necessarily, the overalltransmission rate sij+dij is not increased on the transitions 101.

5B. Delay jitter handling via adaptive packet playout embodimentperforms delay jitter handling using an improvement over S. B. Moon etal., “Packet audio playout delay adjustment: Performance bounds andalgorithms,” ACM/Springer multimedia systems, January 1998. In thisimprovement, overall packet loss rate due to loss, delay, and delayjitter (but before the application of diversity) is used as a QoS levelmeasure.

Transition 101 of FIG. 1 in this 5B embodiment occurs on the criterion(mode=SPIKE)OR(mode=NORMAL AND Overall packet-loss rate>Th1)

Transitions 103 and 105 respectively occur on a criterion (mode=NORMALAND Overall packet-loss rate<Th2). Preferably but not necessarily, theoverall transmission rate sij+dij is not increased on the transitions101.

Here, the rate/diversity control 331 (or alternatively receiver 361′)detects whether network 351 is subject to spike-type delay increaseand/or packet losses (SPIKE) even when the packet-loss rate has not yetexceeded the tolerable threshold Th1, or whether a more smoothly varyingtype of delay change/packet loss behavior (NORMAL) is occurring. Thisinformation is stored as a datum called “mode” for purposes of thisembodiment 5B and used for adaptation. When SPIKE mode is occurring, theembodiment is relatively aggressive, being quick to initiateQoS-enhancing measures, and slow to end them.

One formula recognizes a SPIKE event when magnitude of delay differenceof consecutive packets exceeds twice a variance measure+800 samplingintervals, compare the Moon et al. paper incorporated hereinabove at p.21, Algorithm 2, line 2.

This 5B embodiment herein, however, not only recognizes a SPIKE eventbut also utilizes it for new purposes, processes and structures, toinitiate a SPIKE mode to control a state machine of source rate anddiversity amount. The SPIKE mode herein recognizes that packets arrivewith an average delay based on the time of arrival minus the senderpacket time stamp. Also, the packets have an average jitter magnitude,or measure of variance, in the varying delay values comparing packet topacket. The idea behind SPIKE mode herein recognizes an importantcontrol function for rate/diversity adaptation purposes when themagnitude of delay difference between consecutive packets exceeds somemultiple of the measure of variance plus a constant. The multiplejust-mentioned, reflects the idea that an onset of a significant delaydifference in the incoming packets should be quite substantial comparedto the usual amount of variation in delay in the packet stream. Theconstant reflects the idea that even if the measure of variance wereequal to zero for a packet stream for a while, the onset of some delaydifference would not be important if it were below the amount of theconstant. It should be clear that various formulas and logicimplementations can implement these ideas. One process embodimentdetermines when delay difference of consecutive packets|D(I,I−1)|>2J+800 to initiate the spike mode. Another process embodimentdetermines when delay difference|D(I,I−1)|>mJ+cto initiate the spike mode, where m is a numerical value of a multiplierselected in the range 1.5 to 4 for example, and the constant is equal toaverage measured delay based on timestamps for a last predeterminednumber (e.g. 25) of speech packets.

Another process embodiment uses a logic test to test whether the delaydifference [|D(I,I−1)|>m₁J] OR [|D(I,I−1)|>c1]. m1 is selected in thesame range as numerical value m above. Constant c1 is suitably madesubstantially equal to constant c above.

Still another process embodiment uses any of the foregoing tests butwith an average of delay difference magnitudes to smooth out the processsomewhat, wherein[|D(I+1,I)|+|D(I,I−1)|]/2>mJ+c or alternatively a test[[|D(I+1,I)|+|D(I,I−1)|]/2>mJ] OR [[|D(I+1,I)|+|D(I,I−1)|]/2>c].

Once the SPIKE mode has been initiated, then any one of various testsfor returning to NORMAL mode is implemented. One embodiment repeatedlycomputes a measure of variance and waits until the measure of variancefalls below a predetermined amount, whereupon the NORMAL mode isinitiated. Still another approach utilizes the calculations of the Moonpaper not only for playout delay, but also to derive controls for SPIKEmode and NORMAL mode in a manner that tracks the calculations of SPIKEand NORMAL conditions for playout delay as described in Moon et al.

5CA. A first (herein type 5C embodiment, subtype A) adaptationembodiment with parameters specified in RTCP uses both Fraction Lost andinterarrival jitter field as QoS level measures having their respectivethresholds.

Transitions 101 of FIG. 1 in this 5C embodiment respectively occur onthe criterion(Fraction Lost>Th1)OR(Interarrival Jitter J>Th2)

Transitions 103 and 105 respectively occur on a criterion (FractionLost<Th3) AND (Interarrival Jitter J<Th4)

Note in this 5CA embodiment that QoS enhancing measures are initiated oneither an unacceptable level of Fraction Lost or of Jitter J. However,the QoS enhancing measures are relaxed on the occurrence of BOTHFraction Lost and Jitter J becoming acceptable. Fraction Lost lowerthreshold Th3 is made less than or equal to Fraction Lost higherthreshold Th1. Providing some gap between Th3 and Th1 may help preventoscillations in QoS in some network environments. Similarly, Jitterlower threshold Th4 is made less than or equal to Jitter higherthreshold Th2.

Preferably but not necessarily, the overall transmission rate sij+dij isnot increased on the transitions 101.

5CB. As illustrated in FIG. 22, a second (herein type 5C embodiment,subtype B) adaptation embodiment with parameters specified in RTCP usesboth Fraction Lost and interarrival jitter field as QoS level measureshaving their respective thresholds.

Transitions 101 of FIG. 1 in this 5C embodiment respectively occur onthe criterion(Fraction Lost>Th1)OR(Interarrival Jitter J increases for n consecutiveRTCP reports)

Transitions 103 and 105 respectively occur on a criterion(Fraction Lost<Th2)AND(Interarrival Jitter J is equal to or less thanthe Original Value)

Note in this 5CB embodiment, and in FIG. 22, that QoS enhancing measuresare initiated on either an unacceptable level of Fraction Lost or atrend of a certain number n of consecutive increases of Jitter J. Thenumber n is suitably 5 or any other number accomplishing an effectivecontrol function over QoS. Note that the QoS enhancing measures arerelaxed on the occurrence of BOTH Fraction Lost and Jitter J becomingacceptable. Fraction Lost lower threshold Th2 is made less than or equalto Fraction Lost higher threshold Th1. Providing some gap between Th2and Th1 may help prevent oscillations in QoS in some networkenvironments. Note further that n+1 values of Jitter J are suitablystored in a buffer or window of n+1 Jitter values. If the criterion of nJitter increases occurs, then the oldest of the n+1 Jitter values isstored as the Original Value in a suitable register or memory location.Next the buffer is cleared. New RTCP reports of Jitter J now enter thebuffer one by one, and are each respectively compared with the OriginalValue as stored. When the latest value of Jitter J that has come intothe buffer is less than or equal to the Original Value, and if theFraction Lost is less than Threshold 2 (Th2), then the QoS enhancingmeasures are relaxed by making a transition from state (s22,d22) tostate (s12,d12). Then if the succeeding latest value (or alternativelysome predetermined plural succeeding number n1 of latest values) ofJitter J that has come into the buffer is less than or equal to theOriginal Value, and if the succeeding latest value of Fraction Lost isstill less than Threshold 2 (Th2), then the QoS enhancing measures arestill further relaxed by making a transition from state (s12,d12) tostate (s11,d11) as shown in FIG. 1.

Preferably but not necessarily, the overall transmission rate sij+dij isnot increased on the transitions 101.

It should be apparent that numerous variations on this theme can beintroduced in still further embodiment subtypes.

5D. Adaptation embodiment using TCP throughput estimate for both delayjitter handling approaches compares a ratio of a corresponding TCPthroughput estimate to current overall transmission rate with athreshold. One relatively uncomplicated TCP throughput estimate is givenby: (constant×packetsize)/(round-trip delay×sqrt(average loss measuredduring the lifetime of the connection)). A suitable value of theconstant is 1.22. See D. Sisalem et al., “The loss-delay basedadjustment algorithm: A TCP-friendly adaptation scheme,” NOSSDAV, July1998.

In FIG. 1, this embodiment 5D uses as criterion for transitions 101:(TCP throughput estimate/current overall transmission rate<Th1)

As criterion for transitions 110, 103 and 105 this embodiment 5D uses(TCP throughput estimate/current overall transmission rate>Th2).

“sqrt” means the “square-root function of” Still further, sincethresholding is involved, the throughput estimate and current overalltransmission rate can be squared, and compared with the square ofthreshold Th1 or Th2. This eliminates the square root calculation andspeeds computation in some embodiments.

Preferably but not necessarily, the overall transmission rate sij+dij isnot increased on the transitions 101.

By the use of diversity, the above embodiments handle short-term networkfluctuations well, and cope with VoIP/VOP applications that involvemultiple links of heterogeneous characteristics.

All the QoS level measure computations and adaptation logics aresuitably used for rate/diversity adaptation (rate and diversity both, oreither one alone, selected at different times, or selected on differenttransitions). All the QoS level measure computations and adaptationlogics are suitably used for source rate adaptation alone in variousembodiments without any diversity or diversity adaptation. All the QoSlevel measure computations and adaptation logics are suitably used fordiversity adaptation alone in various embodiments without any sourcerate adaptation.

In FIG. 23, the amount of overall transmission rate reduction and thusseverity of state change advantageously are made to depend on theseverity of congestion. For example, one process embodiment is acombination of the processes of FIGS. 1 and 23. The process of FIG. 1pertains if the congestion severity is low (A≧F>Th1), and a decisionstep chooses the process of FIG. 23 if the congestion severity is high(F>A>Th1). The value A is an adaptation mode threshold value suitably inthe range of 1.1×Th1 to 2.0×Th1. Value of A set at 1.5 times Th1 ismentioned as suitable, for instance. Value A is called an adaptationmode threshold because the mode or process of adaptation (e.g. of FIG. 1and of FIG. 23) is the subject of selection.

Preferably but not necessarily, the overall transmission rate sij+dij isnot increased on the decreasing transitions 2211 and 2215 of FIG. 23.Any of the state transition criteria of embodiments 5A, 5B, 5CA, 5CB,and 5D can be used to trigger a transition. See descriptionshereinabove. For instance if the state transition criteria for F inembodiment 5A is used, then F is a value of overall packet-loss ratereported back in the latest RTCP packet. Similarly, on the returntransitions 2221, 2223 and 2225 then F is a value of overall packet-lossrate reported back in the latest RTCP packet then pertaining to therespective determination step giving rise to the respective transition2221, 2223 and 2225.Overall transmission rates: . . .(s11+d11)=(s12+d12)>(s21+d21)=(s22+d22)>(s31+d31)=(s32+d32) . . . .

Where (d11<d12), (d21<d22) and (d31<d32).

sij denotes the rate for Pn, and dij denotes the rate for Pn′.

Among various embodiments are embodiments for rate/diversity adaptationfor Voice over IP and Voice over Packet. Described herein are systems,integrated circuits, and processes to adapt both rate and diversity, oreach individually, in Voice over IP, Internet Audio, and Voice overPacket (VoIP/VOP) applications. Advantages include a robust solution forhandling network impairments, while utilizing network resourcesefficiently.

As noted earlier, Voice over Packet (VOP) and Voice over InternetProtocol (VoIP) are sensitive to jitter to an extent qualitatively moreimportant than for text data files for example. This sensitivity is alsoa problem for other types of real-time communication media such asframes of compressed video, but for brevity, VOP will be discussed as aplaceholder for the other types of real-time communication as well.

The frame is the data unit for the speech coder. The packet can hold oneframe or more than one frame. With constant number of frames per packet,packet loss rate is equal to frame loss rate.

ATM is a more sophisticated packet network wherein every packet in astream takes the same path, so it represents a form of transmission thatconceptually lies between circuit switching and packet switching. ATM,Frame Relay, and other forms of networking also can benefit by theimprovements described herein.

RTP provides time stamps and packet sequence numbers. UDP (User DatagramProtocol) manages end-to-end transmission without any retransmission.UDP sits in the same layer as TCP. In one embodiment, RTP/UDP/IP isherein utilized for VOP instead of TCP/IP.

When multiple users congest the routers in the network, some packetsbecome lost by actual loss or excessive delay.

In FIG. 4, source rate control with no diversity, an embodiment uses aone-link network having, for example, a hundred users each transmittingat 16 Kb/s. Each packet has a header which takes about 8 Kb/s ofoverhead. As the number of users goes up to 140, for example, the packetloss rate goes to 4%. As the number of users increases, the packet lossrate also increases. In one example of a process embodiment, all theusers are signaled, for instance, by a gatekeeper, to decrease theirtransmit rate from 16 Kb/s to 11.2 Kb/s. Then the packet loss rateadvantageously drops.

The process executes a QoS determination step, which for example is apacket loss rate calculation over a predetermined window interval, givenan expected rate of transmission. For example, if the connectionprotocol has identified a rate of transmission that is high, then ahigher number of packets will be received during the same predeterminedwindow interval at a given QoS than would be received the rate oftransmission is lower. A simple packet loss rate calculation simplymonitors the tags of the packets in a receive buffer, and counts up thenumber of packets that are present (a QoS measure) or those that arelost (Loss Rate which is just an inverse type of QoS measure). If apacket arrives in the buffer with a serial number and/or time stamp thatindicates it is unusable, then it is dropped from the buffer and notcounted. This is because VOP in some forms can only play or decodepackets that have arrived in time to be meaningful to the user.

Another packet loss rate calculation that can be used with a shortreceive buffer keeps independently of the buffer a Service List of thepacket tags and when they were received. The process simply counts fromthe Service List the number of packets which are within the windowinterval (e.g. last 5 seconds), or the number of missing packetsdepending on the approach.

Yet another process increments a counter when a new usable packet isreceived and decrements the counter when the time of arrival of apreviously-counted usable but now-old packets has its time of arrivalhas become prior to the predetermined window interval from the presentinto the past. Other more sophisticated and arithmetically complex QoSmeasures are useful as well.

The skilled worker implements any suitable QoS determination. Forexample RTCP protocol has a reception block with a packet loss ratefield wherein the protocol specification specifies how to compute a QoSmeasure at destination and transmit it back to the source. Thus, onetype of embodiment suitably uses, supports or is compliant with RTCPprotocol. Desirably, the source receives an “effective or overall packetloss rate or ratio” type of QoS measure which takes into account alllost packets, not only those actually lost in the network, but alsothose packets which came too late to be usable for the application, suchas VOP, actually in use at the destination. Note further that theeffective packet loss rate might be less when more sophisticatedinventive VOP application software is implemented at the destination,even though the network congestion were no different.Packets Lost=Packets Lost in Network+Packets Unusable at Destination.EPLR=Effective Packet Loss Ratio=[Packets Lost in Network+PacketsUnusable at Destination]/Packets Sent.

FIG. 3 shows a system embodiment for adaptation to network conditions byadjusting either or both of at least two communications variables, heretransmission rate and diversity. For brevity, process embodimentemployed in this system embodiment is called “Rate/DiversityAdaptation.” The various blocks of FIG. 3 are suitably implemented asall software, all firmware or hardware, or some mixture of software,firmware and hardware allocated and partitioned among the variousblocks. In one embodiment, the blocks are all software code andmanufactured into one or more sections of non-volatile memory on asingle semiconductor chip. Combinations of volatile and non-volatileon-chip and off-chip storage are also suitably implemented in variousother embodiments by the skilled worker.

In FIG. 3, a transmit section 311 in a source computer (computer notshown) has a speech encoder block 321 having sending rate sij. Block 321supplies encoded speech to a Rate/Diversity Adaptation control block331. Control block 331 determines the degree of diversity dij to becommanded by control block 331. Control block 331 feeds a STATE commandto speech encoder 321 to initiate the generation of more or fewerpackets for time-diversity, path diversity or both time and pathdiversity purposes. Also, control block 331 has an Add Diversity portionwhich couples and multiplexes encoded speech from encoder 321 to an RTPPacket Encapsulation block 341. The Add Diversity portion introducesdiversity according to each implemented process embodiment as taughtelsewhere herein depending on STATE. Packet encapsulation block 341supplies, communicates and sends packets to and through a packet network351, to a receive section 361′ of a destination computer (not shown).Receive section 361′ has a Delay jitter Handling block 371′ coupled to aLost Packet Compensation block 381′ which in turn is coupled to a speechdecoder block 391′. One process embodiment operates Lost PacketCompensation block 381′ so that if packet P0 is not received, thenpacket P0′ is decompressed and fed to speech decoder 391′ for playout.Lost Packet Compensation block 381′ in the destination also supplies viaan RTCP packetizer 395′ RTCP packet loss information descriptive ofsource-to-destination packet communication back via packet network 351to the Control block 331 in the source.

Both sides, source and destination, have speech encoder, rate/diversitycontrol block, packet encapsulation, delay jitter handling, lost packetcompensation and speech decoder. Thus, it should be understood that fortwo way communication, there is suitably provided a transmit section311′ (not shown) in the destination computer suitably (but notnecessarily) identical to transmit section 311 described hereinabove.Also suitably provided is a receive section 361 (not shown) in thesource computer suitably (but not necessarily) identical to receivesection 361′ described hereinabove.

Primes in FIG. 3 indicate blocks in the destination computer, andunprimed numerals indicate blocks in the source computer. This format ofdrawing visually and literally communicates the transmitter source pathto the receiver destination. Also, this format concisely andconveniently permits the reader to visualize both the transmit andreceive software blocks 311 and 361 at the transmitter-source end byignoring the primes on the numerals for the receive blocks. Further, theformat represents software blocks 311′ and 361′ at the receiverdestination end by considering all numerals as primed for transmitblocks and receive blocks.

Lost Packet block 381 (not shown) in the source also supplies via anRTCP packetizer 395 (not shown) second RTCP packet loss informationdescriptive of destination-to-source packet communication back viapacket network 351 to the Control block 331′ in the destination.

In other more complex embodiments the path of communication from LostPacket Compensation 381′ to Rate/Diversity control block 331 is suitablymade independent of packet network 351, as by satellite, wireless, PSTN,etc.

Advantageously, control block 331 and compensation block 381 are eachimportant improvements, singly and in combination with each other and incombination with the other blocks described. Also, feeding back STATEcommand information to a speech encoder improved to respond in itsoperations thereto, advantageously confers flexibility and control overQoS under different network 351 loading conditions.

Receive section 361′ has a Delay-jitter Handling block 371′ with abuffer in it, a process for reading the packet headers including theirpacket sequence numbers and time stamps, and a process of discarding orignoring packets that arrive too late. Block 371′ is coupled to a LostPacket Compensation block 381′ which utilizes any suitable means ofreconstructing lost VOP data in lost packets such as by inserting zeroesor white noise, or by interpolation or by reconstructing fromtime-diversity, path diversity, or combined time/path diversity packetinformation.

In addition block 381′ calculates the QoS measure, such as packet lossratio as described earlier hereinabove. Lost Packet block 381′ in thedestination also supplies the RTCP packetizer 395′ the QoS measure whichpacketizer 395′ incorporates into the payload of return RTCP packets andsends them to control block 331.

Block 381′ couples commands and encoded speech data to speech decoderblock 391′. The commands identify which of plural modes speech decoderblock 391′ is to execute. For example, when only a single packet streamhaving a first type of encoding and transmission rate is being received,then the speech decoder is commanded to decode that first type ofencoding and transmission rate. When a single packet stream havingtime-diverse packets in the stream is being received, then the speechdecoder is commanded to decode by type of encoding, and to put thediverse packets information together to somewhat improve the quality ofthe output sound. When multiple packet streams having path-diversepackets are being received, then the speech decoder is commanded todecode by type of encoding, and to put the diverse packets informationtogether to advantageously improve the quality of the output soundaccording to processes particularly emphasized herein. When multiplepacket streams having not only time-diverse packets but alsopath-diverse packets are being received, then the speech decoder iscommanded to decode by type of encoding, and to combine the packetinformation together to further advantageously improve the quality ofthe output sound.

Among various voice coders (vocoders) or speech coders contemplated forblock 321 of FIG. 3 are G.711 PCM (pulse code modulation), 64 kbps(kilobits per second); G.726 ADPCM (adaptive differential pulse codemodulation), 32 kbps; G.729, 8 kbps; G.729 Annex A, reduced complexityversion; G.729 Annex B, silence compression; G.723.1, 5.3/6.3 kbps (dualrate); G.723.1 Annex A, silence compression; TI-CELP and VOP-optimizedvocoders as described in the literature cited herein or elsewhere. Thesespecific identifications of vocoders are non-limiting examples.

Turning again to FIG. 3, the RTCP specification describes feedbackinformation which is contemplated herein as sent from packetizer 395′eventually to rate/diversity control block 331. It is contemplated thatin one type of embodiment the feedback information be computed asdescribed herein and/or as described in the RTCP spec. Note that RTCPcan be extended if an application requires additional feedbackinformation.

In connection with FIG. 9, RTP is useful for real-time data likeinteractive voice and video. RTP services include payload typeidentification, sequence number, time stamp, and synchronization sourceidentifier (identifies sender and any conference contributors to thepacket). The header includes other information, and the payload includesvoice frames, compressed audio, video, real-time control and measurementdata, or other real-time information. The timestamp reflects thesampling instant of the first byte of the first voice frame in thepacket. A clock oscillator in the system of FIG. 3 provides a suitablystable time base for calculating this sampling instant time withsufficient accuracy to allow the delay jitter handling block 371′ andlost packet compensation block 381′ to respond to delay, jitter, andout-of-order packets.

RTP is suitably carried on top of UDP and IP. Each frame or set offrames of audio/voice/video/media has RTP header and a UDP packetcontains the frame(s) and RTP header. The Payload type field in the RTPheader identifies the type of coding that the encoder 321 uses.

RTCP is a control protocol for RTP. An RTCP “report packet” has a headeras in FIG. 20. The report packet carries the synchronization sourceidentifier of each sender 311 that a given one of one or more reportblocks describes, as well as the synchronization source identifier ofthe computer (herein further improved with block 361′) that creates thereport packet. A report block has several fields: 1) fraction lost L, 2)cumulative number of packets lost CL, 3) extended highest sequencenumber received EHSN, 4) interarrival jitter J, 5) last report packettime stamp LSR, and 6) delay since last report packet DLSR. Furtherreport blocks in the report packet of FIG. 20 identify a differentsender computer (as in a conference) and report values L, CL, EHSN, J,LSR, DLSR back to that sender. Profile-specific extensions follow thereport blocks as the skilled worker elects.

In RTCP report packet report block, the Fraction Lost field occupieseight (8) bits. Fraction Lost means the fraction of RTP data packetsthat were lost out of the packets sent by the described-sender since thelast report packet was sent by the reporting sender. Fraction Lost isexpressed as a binary fraction with the binary point at the left edge ofthe 8-bit field. Put another way, the integer occupying the 8-bit fieldis the Fraction Lost multiplied by 256. Put another way, Fraction Lostis the number of packets lost divided by the number of packets expectedduring the period since the last report packet. If the loss is negativedue to duplicates, the fraction lost is set to zero. If all packets arelost in a reporting interval, no reception report is made. Note that invarious alternative embodiments, the Fraction Lost calculation is eitherreplaced by another QoS calculation, or suitably altered so thatduplicates and diverse packets do not decrease the loss fraction.

In FIG. 21, QoS level measure computation process embodiment istemporally localized in order to avoid smoothing of network effects.While other intervals can be used in various embodiments, currently theRTCP Transmission Interval (longer line in FIG. 21) is made long enoughto gather and report statistically meaningful new QoS data. The RTCPtransmission interval is made short enough at receiver 361′ to reportback the new QoS data and also enable the sender 311 to adaptivelychange its source rate and diversity in a manner that is reasonablyresponsive to network conditions and opportunities. The RTCP interval inone range of embodiments is set between 1 second and 30 seconds. Anexample value of 5 seconds RTCP Transmission Interval between the I andI−1 RTCP report packets is contemplated in the foregoing range, forinstance.

The QoS level measure computation process (shorter line QoS in FIG. 21)is preferably arranged to occur right up to the end of the RTCPTransmission Interval so that the latest RTCP report packet is using thelatest QoS data possible. The QoS level measure computation interval ismade short enough at receiver 361′ to avoid smoothing of networkeffects.Packets Lost=Packets Lost in Network+Packets Unusable at Destination.EPLR=Effective Packet Loss Ratio=[Packets Lost in Network+PacketsUnusable at Destination]/Packets Sent.

Number of packets lost in a time interval between reports is suitablycalculated as the difference in the cumulative number of packets lost inthe report packets. The ELSN (extended last sequence number) data in thereport packets is used as follows. Calculate the difference in the ELSNsbetween two report packets to obtain the expected number of packetsduring the interval between the report packets.The packet loss fraction PLR=Packets Lost in Network/#PacketsExpected=(Difference in Cumulative # of Packets Lost)/(Difference inELSNs).

Note that the Effective Packet Loss Ratio includes not only Packets Lostin Network but also Packets Unusable at Destination in the numerator.

In FIG. 20, the value L communicated from receiver to sender is suitablymade equal to EPLR, or alternatively the RTCP Loss Fraction is used.

Interarrival Jitter J is a 32-bit mean deviation, smoothed absolutevalue, of the difference D(I, I−1) in packet spacing at the receivercompared to the sender for a pair of consecutive packets I and (I−1).Difference D is the absolute value of the difference in delays of atleast two received packets. Delay d is the difference between a packet'sRTP timestamp and the time of arrival in RTP time stamp units. Inmathematical terms,Delay d(I)=t(I)−s(I)(actual time received minus time stamp when sampledat source)Delay Difference D(I,I−1)=d(I)−d(I−1)

J is suitably calculated in a calculation loop starting from aninitialized value J of zero and successively calculating:J=J+(|D(I,I−1)|−J)/N. N=16 is an example smoothing divisor constant usedin RTCP.

Other approaches can calculate jitter as an average of absolute valuesof Delay Difference over a window. One procedure, among others suitable,isJ=J+[|D(I,I−1)|−|D(I−N,I−N−1)|]/N. N=16 is an example.

Still other approaches calculate jitter as the statistical variance, orotherwise suitably as the skilled worker elects for the purposes athand.

So another type of embodiment computes jitter J in block 371′ to reportback QoS. Jitter J reported back by RTCP is then compared to a thresholdin rate/diversity control block 331 of FIG. 3. Thus, when packets comeso variably due to jitter that a steady voice decode stream in thereceiver is unobtainable (beyond threshold of acceptability), thenadaptation of rate/diversity should occur.

Still another type of embodiment computes in rate/diversity controlblock 331 a joint function f(Loss Fraction, Jitter) and compares itsvalue f with a threshold and then issues STATE controls based thereonaccording to a control loop similar to that described in FIG. 16, onlywith value f substituted for value L. It should be apparent thatnumerous embodiments of integrated circuits, process, and systems areavailable to the skilled worker in implementation.

Other data in the RTCP report packet are suitably used in fashioning yetother embodiments.

Cumulative Number of Packets Lost is a 24 bit count of lost packetssince beginning of reception.

Extended Highest Sequence Number Received is two data: First, thesequence number in the RTP header of the latest RTP data packet fromsender 311 as received at receiver 361′. Second, a count of sequencenumber cycles.

Last Report Packet Time Stamp is the time of reception at receiver 361′of the latest RTCP report packet received from sender 311.

Delay Since Last Report Packet is the time difference between receptionof an RTCP report packet from the sender 311, and sending this RTCPreport packet from the receiver 361′.

RTCP provides for Profile-Specific Extensions in the report packets.Therefore, various QoS functions as described herein can be computed atthe receiver 361′ in block 371′ and put into the Profile-SpecificExtensions area of the report packet in some embodiments. Otherwise, theQoS functions are suitably computed in block 331 of sender 311 from RTCPreport packet information like Loss Fraction and Jitter coming back fromthe receiver. In still a further variation, embodiments use very shortRTCP application-defined packets, called APP packets and receiver 361′sends back these very short report packets instead of the longer RTCPreport packets. The short APP packet suitably contains Packets Lostonly, from which Loss Fraction is computed at the sender 311. Or theshort packet suitably contains Cumulative Number of Packets Lost only.Or the short report packet from receiver contains Jitter and LossFraction only.

In this way, the introduction of block 381′ as a structural and processimprovement into the system advantageously improves VoIP/VOP quality byestablishing adaptive control of source rate sij and diversity dij.Block 381′ feeds QoS information such as packet-loss rate, delaystatistics and any other information selected by the skilled worker asuseful for this purpose, back to rate/diversity control 331.Rate/diversity control 331 thereupon responds to the feedbackinformation according to any suitable process established in control 331and described herein or hereafter devised to improve QoS when it becomesless satisfactory. Such process can operate according to a thresholdingalgorithm as described elsewherein, or respond in a more gradual mannereither according to more closely spaced thresholds or according to avirtually continuous adjustment of source rate and diversity.

Further, the introduction of block 381′ as a structural and processimprovement into the system advantageously improves VoIP/VOP quality byactually utilizing more of the packets sent to receiver 361′ via packetnetwork 351. Block 381′ utilizes packets having diverse information andcombines their information and controls speech decoder 391′ so as toform a speech output (or other audio or image or other media output)that more nearly replicates the speech input to speech encoder 321originally or otherwise improves the quality.

Having a sender that has RTP protocol and a receiver that has RTCP tofeed back a packet loss fraction to the sender are improved. The senderis improved by introducing rate/diversity control block 331 to adddiversity and rate/diversity adaptation with state feedback to thespeech encoder. Likewise improvements for lost packet compensation forthe receiver are provided by block 381′.

Turning now to FIG. 4, and considering operations at the system level,source rate control with no diversity looks at the packet loss rate, forexample. If the packet loss rate is higher than a particular threshold,then an embodiment of the process requests one, some or all the sendersto reduce their source rate so that the congestion condition can beremoved from network 351.

Packet network 351 is a collection of interconnected routers or nodes,interconnected by links, and for the senders in a complex network, notall of the users are necessarily using the same links between the nodes.Thus, in FIG. 3, user 301.p is using different links such as link 303 incontrast to user 301.q who is using other links such as link 305.However, a link 307 may be used by many users more frequently than someother links in network 351, and thus contributes to packet loss andconsequent low QoS more than do links 303 and 305. Such a link 307 isthen termed a bottleneck link.

In FIG. 4 a bottleneck network link simulation was run with speechactivity at 45%. The header bit rate was set at 8 kbps and the channelcapacity was set for 24×64 kbps (in FIGS. 4, 5, 7 and 8). The simulationstudied source rate control with no diversity. Packet loss rate inpercent (accounting for both actually lost packets and too-late packets)was graphed versus number of users (N) in a family of curves in FIG. 4corresponding to various source rates of 16 kbps, 11.2 kbps and 8 kbpswherein source rate is a parameter of the family of curves. Given aninitial source rate of all users at 16 kbps, then as number of users Nrises, the packet loss rate rapidly and nonlinearly rises along a curve411 until packet loss rate has reached Threshold) of about 4% at about140 users. Further in the FIG. 4 example, requesting a source ratereduction from illustratively 16.0 kbps to 11.2 kbps suddenly andeffectively causes the packet loss rate to fall, as shown by arrow 413,from curve 411 to a curve 421 having parameter 11.2 kbps. Then as moreusers access the network and their number rises to about 180 users, thenthe packet loss rate even on the curve 421 rises rapidly once againnonlinearly to Threshold) of about 4%. Once again, the source rates aredropped for all users, this time from 11.2 kbps to 8 kbps, and thepacket loss rate drops as shown by arrow 423. Now packet loss rate forthe 180 users is near zero, as seen by inspection at N=180 of a curve431 having parameter 8 kbps.

As can be seen from inspection of the FIG. 4, more curves for highersource rates, intermediate source rates, and lower source rates can beadded, and numerous quite sophisticated embodiments can be devised bythe skilled worker to control packet loss rate. For example, the packetloss rate suitably is arranged to fall along an arrow like 413 not toessentially zero as shown but to still-significant positive value, byadding intermediate source rate values and more smoothly adjustingsource rate. Alternatively, the system and process are structured tovary the source rate (and/or diversity) in the manner of aservomechanism or servo process loop to minimize an error defined as thedeparture from a target level of QoS. The loop would lock onto thetarget QoS level, except when to do so would require a source rate inexcess of a maximum source rate permitted for the system or otherwise besubject to some technical constraint. Then arrows 413 and 423 would beessentially insignificant in magnitude.

In FIG. 5, the advantageous effect of diversity in reducing the y-axisresidual packet loss rate is illustrated for a media-specific redundancyexample. Residual packet loss rate accounts only for actually lostpackets and too-late packets that were not compensated by receipt ofdiversity packets. Speech activity is a percentage of time that speechand not silence is occurring. Over a wide span of speech activity from40-60 percent on the x-axis, the residual packet loss rate curve 511when diversity is used, is dramatically lower than for curve 521 when asingle stream of packets is used. Diversity curve 511 is illustrated fors12=11.2 kbps and d12=4.8 kbps using P0P1P′P2P1′ . . . sequence.Diversity curve 521 is illustrated for s11=16 kbps and d11=0 kbps usingP0 P1 P2 . . . sequence. The curves of FIGS. 4, 5, 7 and 8 were derivedfrom a simulation model described in connection with FIG. 12. In FIGS. 5and 7 simulated number of users N was 128. It was assumed that all usersuse the same source rate and diversity rate in computing each curve inFIGS. 5 and 7.

FIG. 7 is similar to FIG. 5 in the axes and uses MD (multipledescription) coding as an example. Further in FIG. 7, residual packetloss rate L is progressively reduced for any given x-axis amount ofspeech activity in the order of curves 711, 721 and 731. Those curvesrespectively represent (sij, dij)=(8,8), (11.2,0) and (5.6,5.6) kbps.Rate/diversity adaptation is dramatically effective.

FIG. 8 illustrates another MD (Multiple Description) coding example.FIG. 8 changes the axis of Packet Loss rate of FIG. 4 to Residual PacketLoss Rate in FIG. 8. No-diversity source rate curves 411 and 421 (16.0and 11.2 kbps source rate respectively) are repeated for clarity in FIG.8 because when no diversity is used Residual Packet Loss Rate equalsPacket Loss Rate. This bottleneck link simulation had speech activityheld constant at 45%, and header bit rate 8 kbps. Again, theintroduction of diversity at a given overall transmission rate (sij+dij)produces a dramatic improvement in residual packet loss rate.

The FIG. 8 curves substantiate the feasibility and advantage of usingstepwise changes of STATE according to a process embodiment of FIG. 1,wherein not only does transition 101 improve QoS but also a further step1007 (of FIG. 10) further improves QoS if such further step becomesneeded. Compare the improvement represented by curve 811 (represents(8,8) case) compared to curve 411 (16,0). At a given number of users theimprovement in packet loss rate, indicated by down-arrow 813 isstriking—down from 3% example Threshold1 to less than 1%. Furthercompare the improvement represented by curve 821 (represents (5.6,5.6)case) compared to curve 421 (11.2,0). At a higher given number of usersthe improvement in packet loss rate, indicated by down-arrow 823 is alsostriking—again down from 3% example Threshold1 to less than 1%.

While in the rate diversity adaptation of FIG. 3 only two users areshown, at a sender 311 and a receiver 361′, various network embodimentsdo contemplate dozens, hundred, thousands or more users of network 351in FIG. 3. Some, many or all of the users are each provided withimproved transmit/receive software and apparatus 311 and 361 of FIG. 3.Packet network 351 is thus recognized to be a common area where theservice demands of many users may start to contend or to producecongestion. Put another way, if FIG. 3 be visualized as the send/receivesoftware 311, 361 and computer of one user, then for 140 users, FIG. 3is replicated or multiplied 140 times with packet network 351 beingcommon to all the diagrams. It should be understood further that becauseof the numerous different embodiments of the invention, various of thoseembodiments are suitably distributed to the various users withadvantageously compatible use of one or more of the various embodimentsall across the network.

In FIG. 3, the RTCP block sends back the packet-loss rate for packetsthat originated at the sender computer for which the receiverdestination computer is receiving. And so, if it so happens that a lowpacket loss rate is detected at RTCP packetization block 395′, thereceiver 361′ will signal back to sender 311 control block 331 a lowpacket loss rate that tells the sender block 331 “do not add anydiversity dij.” Whereas, another receiver at 301.p would detect anotherpacket loss rate from a sender, such as 301.q, from which 301.p isreceiving and not sender 311 and not coordinated in any way withreceiver 361′ either, but independent from both of sender 311 andreceiver 361′. So a receiver at 301.p improved with an embodiment likethat at 361′ would send back packet loss fraction information throughnetwork 351 to its own sender 301.q the instructions whether to adddiversity or not, or the information on which sender 301.q woulddetermine whether to add diversity or not in further transmission to thereceiver at node 301.p.

The amount of source rate adjustment and diversity adjustmentresponsively introduced by a given sender is subject to selection of anyof various embodiment, the selection suitably made by the skilled workerbearing in mind principles of engineering economics, desirably shortresponse-time, and other considerations.

For example, if the RTCP packet loss fraction datum rises above atolerable Threshold1, one type of embodiment makes relatively smalleradjustments to source rate and diversity at the sender, and awaitsreception of one or more additional RTCP packets to determine whetherthe packet loss fraction datum remains above the tolerable Threshold1,whereupon further adjustments to source rate and/or diversity areincrementally introduced until the packet loss fraction has been reducedacceptably.

In another type of embodiment, when the RTCP packet loss fraction datumrises above a tolerable Threshold1, such type of embodiment detects thedifference between the packet loss fraction and Threshold 1 (or comparesthe fraction with Threshold1 and one or more additional even lesstolerable higher Thresholds). Then block 331 in such embodiment makesadjustments to source rate and diversity at the sender, the adjustmentsbeing either incremental or more major depending on whether the packetloss fraction is near Threshold1 or in fact is much greater thanThreshold1. Such embodiment does not await reception of one or moreadditional RTCP packets to determine whether the adjustment should bemajor rather than incremental. However, such embodiment does furtherupdate its adjustments and operations utilizing packet loss fractiondata from further RTCP packets. The process in such embodiment issuitably tuned to produce adjustments that converge upon appropriatelevel of QoS in a desirably short response time. The process is tuned toprevent any major adjustments from leading to oscillation. Oscillationoccurs when major decrease adjustments alternate with major increaseadjustments, or divergent sequences of adjustments happen that do notcontribute to satisfactory QoS.

Sender 311 decides whether to perform rate/diversity adaptationdepending on the particular packet loss fraction (or other QoS measure)reported back from the particular receiver 361′ to which sender 311communications are destined. Sender at node 301.q decides whether toperform rate/diversity adaptation depending on the particular packetloss fraction (or other QoS measure) reported back from the particularreceiver at node 301.p to which the communications from the sender atnode 301.q are destined.

The adjustments thus involve two or more of adjusting source rate sij,adjusting diversity rate dij (involving either or both of time diversityand path diversity rates), and adjusting the overall transmission ratesij+dij.

It is possible to have one receiver 361′ having no problem with packetloss and another receiver say at 301.p having a lot of packet loss.Advantageously, various embodiments avoid contributing in any way to anmetastable or unstable network equilibrium wherein some nodes wouldinteract collectively to “hog” network resources and others would bestarved for network resources. Some schemes use redundancy wherein theyrepeat packets or information therein and thus increase the overalltransmission rate. An important advantage of some embodiments is to usediversity adaptively—in other words, based on the network needs—and toavoid making congestion or the deleterious significance of a givenbottleneck-link worse. Some embodiments have a first advantage of usingthe diversity as it is needed, and/or a second advantage of trying toreduce the congestion by both changing the source rate and the amount ofdiversity as well.

If some senders were encountering a lot of packet loss problems, thensuppose that they start adding diversity and thus attempting to use morenetwork resources and congesting the network some more. A further senderand receiver pair are now forced over their Threshold) of tolerablepacket loss fraction, due to the increase in network congestion due tothe first-mentioned senders. Suppose the further and now newlyover-Threshold) sender and receiver also add diversity and attempt touse more network resources and congest the network still more,introducing QoS problems at further and further senders and receivers,in a snow-balling or chain reaction effect, wherein the packet networkbecomes even more greatly congested leading to network problems. Variousembodiments avoid this problem by adding diversity to recover QoSadaptively, and either use the same overall transmission rate sij+dij,or decrease the overall transmission rate.

The response of sender 311 under control of block 331 might in someembodiments, as illustrated in FIG. 11, ask the network for morebandwidth, and such embodiments can be used where the chain reactionscenario is not applicable. But many of the embodiments advantageouslyavoid the chain reaction scenario even when it is applicable, byadapting to reduce the overall transmission rate sij+dij when the QoSbecomes less acceptable. Thus, a form of the latter such process reducesthe overall transmission rate even when it adds diversity, byconcurrently reducing the source rate by an amount exceeding an amountof contribution due to addition of diversity. In other words, someprocess embodiments are improvements to control the sender in such a wayas not to increase the overall transmission rate, thus the requestedbandwidth and attendant network congestion, but instead will use thenetwork in a way which will reduce lost VoIP/VOP packets.

Rate and diversity usage of the network is optimized subject to theconstraint that overall transmission rate be less than or equal to agiven amount (which might change with network conditions). Inoptimization mathematics terminology, a QoS merit function is optimizedsubject to at least one constraint, namely that overall transmissionrate be less than or equal to a given amount.

QoS is affected jointly by the amounts of source rate at which thespeech encoder is transmitting and given by variable sij and diversitygiven by variable dij commanded by control block 331 of FIG. 3.Diversity has types: time diversity or path diversity or combinedtime/path diversity, for instance.

The overall transmission rate is allocated between the source rate andthe diversity dij.

Given a subsisting state of the network, QoS is a function of sij anddij. Mathematically, QoS is a function of two variables sij and dij.Graphically, QoS is a surface in a three-dimensional space having QoS asa vertical dimension and having two horizontal dimensions sij and dij.Thus,QoS=f(sij,dij).

As between two users p and q, QoS(p,q,t) represents the QoS forcommunications from p to q at time t. QoS(q,p,t) represents the QoS forcommunications in the opposite sense from q to p at time t. QoS varieswith number of users N, speech activity A, and over time depending onthe configuration state of the network. Put another way,QoS=QoS(p,q,sijp,dijp,A,N,t).

This 7-dimensional QoS function expresses the idea that QoS depends onwhich two users are involved, the source rate sij from sender p, thediversity rate dij from sender p, number of users N and the time t.

Let L be the packet loss after the application of diversity, meaningafter any packet recovery or reconstruction that is implemented. Packetloss rate L is inversely related to QoS by a function g(L) soQoS=g(L((p,q,sijp,dijp,A,N,t)).

FIGS. 4, 5, 7 and 8 represent graphs of residual packet loss in varioushyperplanes cutting through the multi-dimensional space representingL(p,q,sijp,dijp,A,N,t).

Further, the following inequality expresses the challenge solved byembodiments herein to keep QoS above a threshold subject to a constrainton each sender p on overall transmission rate:QoS(p,q,sijp,dijp,A,N,t)>thresholdsijp+dijp<=max overall transmission rate respective to each sender p.or, put otherwise,L(p,q,sijp,dijp,A,N,t)<thresholdsijp+dijp<=max overall transmission rate respective to each sender p.

The sender takes advantage of the diverse properties of the packetnetwork so as to reduce the packet loss rate and increase the QoS to atleast an acceptable amount.

When the threshold applies to all the users on the network, then thechallenge of maintaining high QoS calls for the network-friendlyadvantages discussed elsewhere herein.

Some embodiments have just one QoS threshold wherein if the transmissionquality becomes less acceptable than that threshold, an embodiment willmake an adjustment in rate/diversity adaptation indicated by transition101 of FIG. 1 to improve QoS. However, the fact of even a singlethreshold does not prevent control block 331 from making one or morefurther adjustments like transition 1007 of FIG. 10 when additional RTCPpackets indicate that the QoS remains on the unacceptable side of thethreshold. The apparatus acts like a servomechanism wherein the use ofeven just one error threshold (although other embodiments herein can usemore thresholds) enables the servo notwithstanding to continually makeadjustments in its output to keep the error within the threshold. Thus,when QoS is unacceptable, the apparatus keeps adapting, if possible, bychanging the values of sij, dij, or both, until QoS is brought within anacceptable range. At some point, given a subsisting state of thenetwork, there will be an state of adaptation of an embodiment thatconfers not just adequate QoS but optimum QoS. Some embodiments find abarely acceptable QoS, others provide adaptive ways to reach the optimalQoS structure of (sij, dij) and search until the optimum QoS isobtained. Suppose the optimal QoS is obtained in the computers of everyone of the network users. Since the optimum is the best the apparatuscan do under constrained circumstances, consider whether networkperformance will start to degrade or become unstable. When circumstancesare bad, the conditions are improved by adding diversity and reducingthe overall transmission rate. When the situation is measurablyimproved, the source rate is suitably increased cautiously orincrementally through multiple stages 103 and 105 in FIG. 1, while thediversity is suitably reduced, maintained, or in the example oftransition 103 even increased. Further in the example, the diversity isreduced or terminated on the final recovery transition 105 in oval 111.

Control block 331 takes original compressed speech packets P0, P1, P2,P3 and suitably adds diversity. If diversity robustness becomes needed,then it adds P0′ a form of the information in packet P0. So if thenetwork loses the packet P0, or P0 comes too late and thus is notavailable while P0′ is on hand, then the lost packet compensation block381′ uses bits P0′ to reconstruct the information in packet P0 to someextent or even fully.

In FIG. 6 no diversity is implemented as a stream of packets P1, P2, P3with their respective different headers H. Time diversity, in a firstexample has packet 611 with the information of packet P1 and anappended, or trailing set of compressed bits (0) having informationdependent relative to a previous packet P0. Packet 611 is followed bypacket 613 bearing information of packet P2 and an appended, ortrailing, set of compressed bits (1) having information dependentrelative to packet P1. Packet 613 is similarly followed by packet 615bearing information of packet P3 and a trailing set of compressed bitshaving information dependent relative to packet P2.

Time diversity, in a second example has packet 621 with the informationof packet P1 and trailing bits for compressed information dependentrelative to two previous packets. Packet 621 is followed by packet 623bearing information of packet P2 and an appended, or trailing, set ofcompressed bits having information dependent relative to packet P1 andthe next previous packet before P1. Packet 623 is similarly followed bypacket 625 bearing information of packet P3 and two trailing sets ofcompressed bits having information dependent relative to packets P2 andP1 respectively. Succeeding packets have information of the nth packetand two trailing sets of compressed bits having information dependentrelative to the nth packet's two predecessor packets P(n−1) and P(n−2).

Further in FIG. 6, a packet 631 having the information of packet P1 issent through the network. Further, a packet 632 has compressed bitswhich are a function not only of packet P1 information but also packetP2 information. Further in the sequence, packets 635 and 637 aresuccessively sent bearing the information of packets P2 and P3.Dependent packet 636 bears information that is a function f(2,3) ofpackets 635 and 637. Dependent packet 638 is similarly constructed insequence. Here an example of function f is exclusive -OR (XOR) as in FECparity schemes.

Another embodiment of time diversity has packets 641, 643, 645, etc.having information same as packets P1, P2, P3, etc. as well asrespective appended information bits dependent as a joint function ofthe information in the two preceding packets, e.g. f(1,2).

In FIG. 11, the source rate s11 of a packet stream 1111 is a givenamount. When QoS falls, the control block 331 utilizes an alternativeprocess wherein it reduces the source rate to an amount s22 compared tofirst source rate s11. However, diversity d22 is added in packet stream1121 and the overall transmission rate s22+d22 exceeds the originaltransmission rate s11 (where d11 was zero).

In a further alternative, control block 331 maintains source rate s22equal to s11 and diversity d22 is added as shown in a packet stream1131. Here again overall transmission rate exceeds the originaltransmission rate of stream 1111.

FIG. 12 illustrates packet loss simulation. Voice sources 1, 2, 3, . . .N each have two-state Markov models comprising speech and silencestates. Each voice source is assumed to use the same coder rate R. Thevoice sources are fed to a buffer 1211 and thereupon to a communicationslink 1221 having a capacity C of, for example, 24×64 kbps. The buffer1211 has a size, for example, of C/(R+H) packets, where C is linkcapacity, R is coder rate, and H is overhead rate. The model simulatesto determine the packet loss rate L which results from variouscombinations of source rate, time diversity and path diversity, thusproducing the graphs of FIGS. 4,5,7 and 8. Software prepared in astraightforward manner by the skilled worker operates various blocks ofthe system of FIG. 3 and implements the process embodiments such asthose represented by the state transition diagrams of FIGS. 1 and 10.

In FIG. 13, input to the simulator of FIG. 12 varied number of users Nover a time interval of ten (10) minutes (600 seconds). N started out at128, rose to 155, then dropped to 112, rose to 164 and then dropped to120. The simulator of FIG. 12 then produced packet loss rate data. Thisdata was fed to control software for control block 331, which in turnproduced states of FIG. 1 for line STATE in FIG. 3 in order to bringpacket loss rate, and thus QoS, under control.

In FIG. 14, the states produced by the control block 331 are illustratedin a graph of Overall Transmission Rate sij+dij versus time t over theten minute interval of FIG. 13. During use by the initial number N=128of users, 16 kbps source rate and zero diversity is selected by controlblock 331, corresponding to state (s11,d11) of FIG. 1. When the usersrise to N=155, the control block downshifts to 8 kbps source rate and3.2 kbps of diversity, corresponding to FIG. 1 transition 101 to state(s22,d22). Then a transition next occurs after about 5 seconds to astate (s21, d21) of (11.2, 0) kbps which persist for about 100 seconds.Next, when the users drop to N=112, the control block 331 transitionsback to 16 kbps overall transmission rate, but does the transition astwo-step up-shift in the structure of source rate and diversity asfollows. First, the transition goes to state (s12,d12) using 11.2 kbpssource rate and 4.8 kbps diversity rate. Second, a succeeding transitiongoes from state (s12,d12) back to state (s11,d11) and recovers 16 kbpssource rate and turns off the diversity to zero. Later and further inFIG. 13, when the number of users subsequently goes to N=164 and thenlastly down to N=120 in FIG. 13, the control block 331 adapts by againtransitioning through steps as described.

In FIG. 15, a packet voice digital signal processor (DSP) is implementedas an integrated circuit 1411. The integrated circuit is suitably a CMOSDSP such as any suitable selection from the TMS320C54x or TSM320C6x DSPfamilies, or other such families commercially available from TexasInstruments Incorporated, Dallas, Tex. USA. See Wireless andTelecommunications Products: Central Office, Telemetry RF Receivers andPersonal Communications Solutions, Data Book, Texas InstrumentsIncorporated, 1996, which is hereby incorporated herein by reference,and particular Chapter 9, Digital Signal Processors therein.

For example, the TMS320C54x fixed-point, DSP family is fabricated with acombination of an advanced modified Harvard architecture which has oneprogram memory bus and three data memory buses. This processor alsoprovides a central arithmetic logic unit which has a high degree ofparallelism and application-specific hardware logic, on-chip memory,additional on-chip peripherals. This DSP provides a specializedinstruction set for operational flexibility and speed of the DSP.

Separate program and data spaces allow simultaneous access to programinstructions and data. Two reads and one write operation can beperformed in a single cycle. Instructions with parallel store andapplication-specific instructions are provided. Data can be transferredbetween data and program spaces. The parallelism supports a powerful setof arithmetic, logic and bit-manipulation operations that can all beperformed in a single machine cycle. Control mechanisms manageinterrupts, repeated operations and function calling. On-chip RAM andROM memories are provided. Peripherals on-chip include serial port andHPI host port interface.

In FIG. 15, integrated circuit 1511 is improved with softwaremanufactured into the ROM, or other nonvolatile, memory for implementingsome part of the process embodiments. Thus, FIG. 15 emphasizes anexample of software blocks manufactured into the integrated circuit1511, the hardware described hereinabove being understood. Thus,description in software parlance follows next regarding FIG. 15 whereinfor example a “unit” refers primarily to a block of software, although ahardware block is another suitable alternative.

In FIG. 15, voice samples are supplied from an analog to digitalconverter (ADC) not shown, to a PCM interface 1515 and converted thereto pulse code modulation. Next the PCM is fed to an Echo Canceller block1517, which feeds a Gain Control block 1521. Gain control 1521 suppliesa Voice Activity Detector 1531 which detects whether voice packets orsilence packets are to be generated. The output of Voice ActivityDetector 1531 goes to a speech coder 1541 having a Voice Coding Unit, orencoder, 1551. The speech coder 1541 is suitably devised or implementedby the skilled worker so as to have multiple coding rate modes ascontemplated herein. For one example, G.729 and Annexes with 11.8 kbps,8 kbps and 6.4 kbps selectable source rates sij is suitably used. Thenan Add Diversity (dij) Rate/Diversity Control Block 1561 couples theoutput of encoder 1551 to a Packet Encapsulation Unit 1571 whichthereupon outputs voice packets from the DSP. Control Block 1561 alsofeeds back STATE (sij,dij) control signals back to voice coding unit1551 to command unit 1551 to produce speech packets at the sij sourcerate, and to produce diversity packets for diversity transmission atrate dij via packet encapsulation unit 1571.

On a receive path in FIG. 15 voice packets enter packet encapsulationunit 1571 where they are depacketized and passed to a Packet PlayoutControl Unit 1581. Control Unit 1581 has software that implementsprocess steps for delay handling, delay jitter handling and lost packetcompensation. Incoming RTCP packets contain lost packet fractioninformation from a destination across the network external to integratedcircuit 1511. This lost packet fraction information is fed via a path1583 to the Rate/Diversity control block 1561.

Also, the delay and jitter handling portion of Packet Playout ControlUnit 1581 includes software process steps to produce second lost packetfraction information representative of the incoming voice packets tointegrated circuit 1511. This second lost packet fraction information isfed via a path 1585 to Packet Encapsulation Unit 1571 which packetizesthe second lost packet fraction information into outgoing RTCP packetsto update the destination across the network. The destination issuitably improved with an integrated circuit 1511′ (not shown) similarto or identical to integrated circuit 1511 of FIG. 15.

From Packet Playout Control Unit 1581, depacketized compressed voiceinformation being received is then supplied in a controlled manner to aspeech decoder 1555 portion of speech coder 1541. Silence packets andvoice packets, suitably dejittered and compensated by use of diversitypackets as improved according to any of various process embodimentsherein, then are decoded by speech decoder 1555 and thus played out. Thespeech thus played out, passes via Gain Control 1521 to PCM interfaceand from there to a DAC (digital to analog converter) not shown whichcan be provided either on-chip or off-chip as the skilled worker elects.The PCM output as converted by the DAC thus reconstitutes the voice inan advantageous manner more fully satisfactory and enjoyable to theuser, by virtue of the various improvements provided and discussedherein. Further, a DTMF “touch-tone” generator 1591 and Tone Detector1593 handle the dialing steps for placing a VOP/VoIP telephone call toconfer a comprehensive application improved as discussed herein.

Operations of add diversity block 1561 of FIG. 15 are illustrated inFIG. 16. A software implementation is illustrated. In FIG. 16,operations commence at BEGIN 1601 and proceed to a step 1605 toinitialize a vector STATE having vector element values s (source rate)and d (diversity rate). The values s and d are initialized with initialvalues s11 and d11 respectively.

Next in FIG. 16, a loop begins at an input step 1611 to input anewly-arriving QoS datum such as a new RTCP report QoS inverse measuresuch as the packet loss fraction L. If no RTCP packet is present,operations branch to a RETURN 1614. If an RTCP packet is present, thenoperations proceed to a decision step 1615.

In the decision step 1615, the value L is compared to determine if Lexceeds a first threshold, designated Threshold) or Th1, indicating toomuch packet loss at the destination. If too much packet loss, thenoperations proceed from step 1615 to a decision step 1617 to determinewhether L also exceeds an even higher level A. If L is less than orequal to A, operations go next to a moderate update step 1621. If Lexceeds A, operations go to an aggressive update step 1623.

In step 1621, a vector NEWSTATE is moderately updated in the mannershown in FIG. 1 and intended to improve QoS and likely reduce value Lexpected subsequently when the destination reports back. NEWSTATE is anintermediate state value holding vector, used intermediately incontrolling the values called STATE. NEWSTATE is suitably updatedaccording to any software method selected by the skilled worker, such asby looking up in a table, or executing a CASE statement, or implementinga software state machine, or otherwise. If the source rate can bedecreased no further, and the diversity can be increased no further,then step 1621 simply makes NEWSTATE the same as the current state.After step 1621 operations go to step 1651.

In step 1623, vector NEWSTATE is aggressively updated (for example tostate (s32,d32)) in the manner shown in FIG. 23 and intended to improveQoS and likely reduce value L expected subsequently when the destinationreports back. Here again, NEWSTATE is suitably updated according to anysoftware method selected by the skilled worker, such as by looking up ina table, or executing a CASE statement, or implementing a software statemachine, or otherwise. If the source rate can be decreased no further,and the diversity can be increased no further, then step 1623 simplymakes NEWSTATE the same as the current state. After step 1623 operationsgo to step 1651.

Some embodiments have thresholds B, C, etc., such that NEWSTATE ismoderately updated if (A≧F>Th1), NEWSTATE is aggressively updated if(B≧F>A>Th1), and NEWSTATE is even more aggressively updated if(F>B>A>Th1).

If in step 1615, L does not exceed Threshold1, operations proceed to adecision step 1627. Step 1627 determines if either the gatekeeper signalGK is on (GK=1) OR a buffer full flag is on (BFR=1). If NO, thenoperations go directly to step 1625, but if YES, operations go to anupdate step 1629.

In step 1629, vector NEWSTATE is updated (for example to state(s31,d31)) in the manner shown in FIG. 23 and intended to improve QoSand likely reduce value L expected subsequently when the destinationreports back. Here again, NEWSTATE is suitably updated according to anysoftware method selected by the skilled worker, such as by looking up ina table, or executing a CASE statement, or implementing a software statemachine, or otherwise. After step 1629, operations go to step 1651.

In decision step 1625, the packet loss fraction value L is compared witha second threshold Threshold2, or Th2. If value L is less than Th2, thenQoS has improved or is at a high level already. In such case, operationspass to a step 1631 to update NEWSTATE to increase the source rate. Step1631 inputs a new estimated steady state overall transmission rate S. Asin steps 1621 and 1623, NEWSTATE is suitably updated according to anysoftware method selected by the skilled worker, such as by looking up ina table, or executing a CASE statement, or implementing a software statemachine, or otherwise. As indicated in FIG. 1 and FIG. 10, the processneed not be simply the reverse of the transitions available in step1621, and so step 1631 is customized for its own updating purposes.Also, if the source rate can be increased no further, and the diversitycan be decreased no further, then step 1631 simply makes NEWSTATE thesame as the current state. One embodiment performs aggressive overalltransmission rate increase if the ratio R of estimated steady stateoverall transmission rate S to current overall transmission rate exceedsa threshold Th3 (e.g. 3.0), and performs gradual overall transmissionrate increase otherwise. For example, such embodiment uses TCPthroughput estimate for new estimated steady state overall transmissionrate S. The TCP throughput estimate is suitably that given earlier (5D)as 1.22×packetsize/(round-trip delay x sqrt (average loss measuredduring lifetime of the connection)).

After step 1631 operations go to a step 1651. If in step 1625, valuewere greater than or equal to second threshold Th2, then operations goto decision step 1635.

In decision step 1635, the packet loss fraction value L is tested to seeif it lies in the range from first threshold Th1 to second threshold Th2inclusive. If yes, then operations go to a step 1641 whereinintermediate NEWSTATE is filled with the values in the vector STATE.Together with a later step 1651, this operation 1641 maintains thecurrent control state. If the decision in step 1635 is NO (out ofrange), or when step 1641 is completed, then operations pass to step1651.

In step 1651 the vector STATE is filled with the values of NEWSTATE.Next in an output step 1661, the values of STATE are output as controlsignals (sij, dij) to the encoder 321 of FIG. 3 and the encoder 1551 ofFIG. 15. Steps 1651 and 1661 thus implement transitions like 101 and 413(see FIG. 4) and other transitions discussed herein. From step 1661operations pass to a decision step 1671 whether to STOP. If not, thenoperations loop back to step 1611 and continue repeatedly as discussedabove. If STOP is yes, then operations go to END 1681. STOP may beresponsive to disconnection of communications with the particulardestination, or to power off or to other conditions of a chip or systemas desired.

The herein-incorporated patent U.S. Pat. No. 6,496,477 provides furtherdisclosure about how path diversity packets are added. The emphasis ofFIGS. 1,3, 15, 16, 17 and 18 are on the adaptive control features ofsome process, integrated circuit and system embodiments whereby sourcerate and diversity are either individually or jointly initiated,increased, decreased and terminated. The said adaptations are performedaccording to a process embodiment in response to QoS-related dataobtained from the network or from a destination monitoring process. Theadaptively-determined sij and dij, or controls generated in arelationship to sij and dij in the manner of a function thereof orsubstantially correlated to them, are then used to start, stop andadjust the operations of the diversity software and hardware disclosedin herein-incorporated U.S. Pat. No. 6,496,477 and the other disclosureherein.

In FIG. 17, system components are arranged to provide gateway functionsand combined with cellular phone base-station functions. A communicationsystem 1701 interfaces to a PSTN (public switched telephone network)1703, to a telephone 1705 (and PBX private branch exchanged connected tomany wired and cordless telephones, not illustrated), to a fax machine1707 and to cellular telephones 1709. PSTN 1703 is coupled via T1/E1Framer 1711 to a DSQ Switch 1741. Telephone 1705 and Fax 1707 arecoupled via a PCM Codec 1721 to the DSQ Switch 1741. Cellular telephones1709 are coupled via a wireless communications interface 1731 to the DSQSwitch 1741.

Further in FIG. 17, the DSQ switch 1741 couples the various types ofcommunications to a first port of a bank of one or more DSPs (digitalsignal processors, such as TI TMS320C6x or TMS320C54x DSPs) 1751, 1753,and so on to the Nth DSP 1755 in the DSP bank. Each DSP suitably hasassociated memory 1761, 1763, . . . 1765 respectively provided as anysuitable mix of volatile and nonvolatile memory selected by the skilledworker. The DSPs are connected via a second port of the bank to a bus1771 which couples them to a microcontroller 1781 that has its own RAMmemory 1783 and flash nonvolatile memory 1785. The microcontroller 1781communicates via a PHY, or Network Physical Interface 1791, to packetdata network 351 of FIG. 3.

In FIG. 17, one, some or all of the DSPs are improved for adaptiverate/diversity operation as described herein. Also, various parts of theimprovements described herein are suitably partitioned between the DSPs1751, 1753, . . . 1755 and the microcontroller (MCU) 1781 and storedon-chip and in the off-chip memories as desired. Various partitioningalternatives are contemplated. Also, the MCU is omitted in anotherembodiment (not shown) and the various software blocks are partitionedamong execution units of one DSP or among multiple DSPs.

In FIG. 18, the improvements are illustratively partitioned so that theRTCP is associated with MCU 1781 of FIG. 17 and the rate/diversitycontrol block 331 and lost packet compensation block 381 (not shown butunprimed in sender 311 of FIG. 3) are provided in the DSP softwarecomplement.

In FIG. 18, MCU 1781 of FIG. 17 is provided with a TCP/UDP/IP stack 1811which further has MAC/ARP, Ethernet driver and other network interfaceprotocol blocks. Further, network management software 1815 for MCU 1781has a network management agent controlling and interfacing to a firstsoftware block for embedded webserver HTTP (Hypertext Transfer Protocol)and Java applications, a second software block for SNMP protocol, VoiceMIBs, and Protocol MIBs, and a third software block for TFTP softwaredownload. Still further, telephone signaling gateway software for MCU1781 has call processing software, address translation and parsingsoftware, and H.323 protocols including H.225 signaling, H.245 software,and RAS/RTCP software. The RTCP function in block 1819 is coupled to theUDP function in TCP/UDP/IP stack 1811 and also coupled to the PacketEncapsulation unit in DSP 1511.

A DSP interface manager software block 1821 is coupled to softwareblocks 1811, 1815, 1819 and 1823 and communicates with DSP 1511 and thesoftware blocks described in connection therewith.

MCU 1781 runs system software 1823 including RTOS (real time operatingsystem such as Microsoft Windows CE or Symbian EPOC, as well as DSPBIOS™ RTOS from Texas Instruments Inc.) System software 1823 includesWDT driver software, flash memory manager, BSP software, development andself-test (IPQST) software, and software installation code.

DSP 1511 has software in FIG. 18 improved as described in FIG. 15 foradaptive rate/diversity in both the send and receive functions.

In other embodiments, as shown in FIG. 19, network 351 has cellularphone base stations 1911, 1913, 1915, 1917. Cell phone base stations1911, 1913, 1915, 1917 are improved to be multimodal, receivinguser-selected packet voice or non-packet wireless voice from cell phones1921, 1923, 1925, 1931, 1933, 1935, 1937, 1939. Wireless two-waycommunications are established between pairs of units listed as orderedpairs (cell-phone, base station): (1921, 1915), (1933, 1915), (1925,1911), (1935, 1911), (1923, 1913), (1931, 1913), (1937, 1917), (1939,1917).

Some of the cell-phones 1921, 1923, 1925, and 1937 have a shadedrectangle, indicating for purposes herein improvements for adaptiverate/diversity as disclosed herein in their packet voice communicationsmode, for example as shown in any one or more of FIGS. 1, 3, 15-18.Other illustrated cell-phones 1931, 1933, 1935, 1939 lack theimprovements for adaptive rate/diversity as disclosed herein in theirpacket voice communications mode, and have no corresponding shadedrectangle in FIG. 19.

Some of the cell phone base stations 1911, 1913 have a shaded rectangle,indicating for purposes herein improvements for adaptive rate/diversityas disclosed herein in their packet voice communications mode, forexample as shown in any one or more of FIGS. 1, 3, 15-18. Otherillustrated base stations 1915, 1917 lack the improvements for adaptiverate/diversity as disclosed herein in their packet voice communicationsmode, and has no corresponding shaded rectangle in FIG. 19. Even when noadaptive source rate/diversity control feature is provided, as in thebase stations 1915 and 1917, they do support both the mobile InternetProtocol phones (IP-phones) wherein IP-packetization suitably occurs atthe mobile IP-phone, as well as support conventional wireless mobilephones.

Personal computer (PC) telephony units 1951 and 1953 have respectivemicrophones and speakers, and these units 1951 and 1953 have modems ofany suitable type, such as voice-band V.90, DSL (digital subscriberline), cable modem, wireless modem, among other choices. Personalcomputer (PC) telephony units 1951 and 1953 are respectively coupled tothe network 351 via gateways 1961 and 1963 respectively. The gatewaysare suitably located in a private branch exchange or in a telephonecentral office, or in the office of an ISP (Internet Service Provider)or in the office of a private commercial network, for example.IP-packetization occurs at the PC telephony units 1951 and 1953. Theadaptation is end-to-end such as when phone at source has therate/diversity control block and the phone at destination has a block tosend QoS data back as well as to couple diverse packet information tothe decoder for improved QoS.

For placing phone calls over the Internet, user voice goes in throughmicrophone, then is processed in the computer by the mainmicroprocessor, microcontroller, and DSP for vocoding and rate/diversityadaptation as in FIG. 3. Even if the access of the PC telephony unit isconnected via voice-band modem to telephone central office and then toInternet service provider, the rate/diversity adaptation software issuitably provided in the PC telephony unit or wherever therate/diversity adaptation software is suitably installed to adaptivelyproduce diversity packets or dependent packets and/or to control thestate of a voice encoder or audio compressor or image compressor orother media coder.

Adaptive rate/diversity improvements in an integrated circuit, softwareand system are suitably provided in an Internet mobile terminal such asan Internet appliance or mobile phone, cell phone or cordless phone withInternet or other packet network capability.

Cell phone base stations 1915, 1917 and 1911 are respectively coupled toIP packet network 351 via PSTN blocks 1971, 1973 and 1975 respectively.Each of the PSTN blocks 1971 and 1973 has a gateway therein to connectthe call to the packet network 351. The gateways in PSTN blocks 1971 and1973 suitably have adaptive rate/diversity embodiments included therein.Thus, rate/diversity adaptation modules suitably are sited in thegateways and base stations of the system of FIG. 19. For example, basestations 1911 and 1913 are directly connected to packet network 351 bytheir own adaptive rate/diversity packet interface software and softwarestacks, all as taught herein in the present patent application and theincorporated U.S. Pat. No. 6,496,477.

A gateway GW 1981 couples network 351 to PSTN 1983 to which telephones(not shown) are coupled via a PBX 1985. Also, one or more individualtelephones 1987 are directly connected to PSTN 1981. Further in FIG. 19,a LAN has nodes 1991 and 1993 coupled to network 351. A computer 1995 isconnected to node 1993.

Integrated circuits into which the adaptive rate/diversity improvementsare suitably manufactured include DSP (digital signal processor) fromTexas Instruments and other companies offering DSP integrated circuits.Other integrated circuits suitable for the adaptive rate/diversityimprovements include host microprocessor such as Intel's Pentium®,Pentium II®, Pentium III®, Celeron®, Xeon® and IA-64 microprocessors,AMD K6 and K7 microprocessors, National MediaGX and othermicroprocessors, and microcontrollers such as ARM and StrongARM series,MIPS series, Intel i960, Motorola Mcore and PowerPC integrated circuits,among many others. Still other integrated circuits which arecontemplated for adaptive rate/diversity improvements includenonvolatile memories such as ROM, EPROM, EEPROM, Flash memory, EAROM,and FeRAM (Ferroelectric random access memory). Volatile memories suchas DRAM, synchronous DRAM (SDRAM), R-DRAM (Rambus DRAM), DDR-DRAM, andother variants suitably also have a logic section or non-volatilesection incorporating the adaptive rate/diversity improvements builtinto them as taught herein. In yet other embodiments, the adaptiverate/diversity improvements are loaded onto or manufactured into rigiddisk drives, hard disk drives, and also various media such as floppyinsertable disks, CD-ROM optical storage media, and/or chips in the readcircuitry or other circuitry of drives for such storage media. Also,chipsets associated with processors suitably are in improvementembodiments made to have adaptive rate/diversity improvementsmanufactured into them, such as the Intel “440xx” series of chipsets,sometimes known as North Bridge and South Bridge chips, and the chipsetsof other chipset manufacturers. (Chipsets of this type are also suitablyimproved with digital signal processors, as taught in any one or more ofthe patents U.S. Pat. No. 5,987,590 and U.S. Pat. No. 6,179,489 whichare hereby incorporated by reference. The DSP suitably runs the adaptiverate/diversity improvements. In other versions, the adaptive softwareruns on the host microprocessor such as Pentium series or IA-64 series,or partitioned with part of the software on a DSP coupled to the hostmicroprocessor(s) in the computer system.

Also, the rate/diversity adaptation can be provided at the telephonecentral office gateway. Also, rate/diversity adaptation can be put in arouter in a packet network to improve it there.

Even more advantageously, when the adaptation is end-to-end and theunits at both ends have at least the adaptation software, the mobilephone or desktop or notebook PC telephony unit adapts for advantageouslysatisfactory QoS.

As discussed further, an improved cell-phone base station (and also agateway improved similarly) runs multiple packet voice modules with aninventive embodiment for each mobile telephone using the base station ata given time.

Some improvement embodiments are intended for gateways, wherein aimproved gateway embodiment runs multiple packet voice modules with aninventive process, chip and system embodiment working in the gatewayitself to adaptively control source rate and diversity rate foradvantageous QoS for each telephone using the gateway at a given time.

In other embodiments, a base station itself is not only improved to bemultimodal, supporting both the mobile Internet Protocol phones(IP-phones) and conventional mobile phones. But also, the improved basestation embodiment runs multiple packet voice modules with an inventiveprocess, chip and system embodiment working in the base station itselfto adaptively control source rate and diversity rate for advantageousQoS for each cell-telephone communicating speech in non-packet form tothe base station at a given time. Then the base station itself and notnecessarily the cell-phone codes or recodes the speech and packetizes itwith an inventive process, chip and system embodiment working toadaptively control source rate and diversity rate for advantageous QoSover a packet network to which the base-station is in turn connected.

Numerous combination embodiments and paths of advantageous operation areconveniently identified in FIG. 19 using sequences apparatus numerals toname them. For example, a communication path 1921-1915-1961-1951 has theimproved adaptive VOP in the terminal handset or PC IP phones but not inthe base station or gateway. Path 1935-1911-1913-1931 has improved VOPin the base stations but not in the terminal handsets. Robustly, stillother paths and embodiments like 1925-1911-1913-1923 have improvedadaptive VOP both in the terminal handset and in the base stations. Herethe adaptive VOP blocks work together or one defers to another as theskilled worker suitably elects to implement. The adaptive VOP blocks areadvantageously upwardly compatible, so that an improved elements 1921,1911, 1951, or 1963 for some examples, can talk to unimproved elementssuch as 1933, 1917, 1961 or 1939 and vice versa. Still othercombinations and paths are present in FIG. 19 and important to peruse,but for conciseness do not appear to need tedious further explanation.

FIG. 20 shows a RTCP packet as discussed earlier hereinabove.

FIG. 21 shows QoS processing timing as discussed earlier hereinabove.

FIG. 22 shows a state transition diagram of a state diagram forhereinabove-described Type 5CB QoS level measure computations andAdaptation Logics.

In FIG. 23, a state diagram for rate/diversity adaptation process andapparatus has example states (sij, dij) as already discussed in FIG. 1.In FIG. 23, however, the state transitions operate differently. Whenoperations are in state (s11,d11), e.g. (16.0, 0.0) and criterion F notonly exceeds Threshold) but also aggressive trigger level A, then atransition 2211 goes from (s11,d11) to state (s32,d32), e.g., (5.7,2.3)kbps of source rate and diversity rate respectively. Another transition2215 is discussed later hereinbelow. After transition 2211, operationsremain at state (s32,d32) unless and until criterion F becomesameliorated and falls below Threshold2, whereupon a state transition2221 to state (s22,d22) e.g., (8.0,3.2). By transition 2221, source rateis increased, and diversity rate is also increased, and their sum(overall transmission rate) is increased. Operations remain at state(s22,d22) unless criterion F continues to be below Threshold2, or incase criterion F rises and later falls below Threshold2. Thereupon astate transition 2223 transfers the system to state (s12,d12) e.g.,(11.2,4.8). By transition 2223, source rate is increased, and diversityrate is also increased, and their sum (overall transmission rate) isincreased. Operations remain at state (s12,d12) unless criterion Fcontinues to be below Threshold2, or in case criterion F rises and laterfalls below Threshold2. Thereupon a state transition 2225 transfers thesystem to state (s11,d11) e.g., (16.0,0.0). By transition 2225, sourcerate is increased, but diversity rate is decreased to zero, and theirsum (overall transmission rate) is maintained unchanged at 16.0 kbps.Transition 2225 thus increases source rate and turns off the diversityfeature. This turnoff is suitably accomplished in some embodiments byterminating the path diversity connection, and suitably accomplished inother embodiments by holding the path diversity connection open forinstant use in case another transition 2211 is needed, but nottransmitting any voice packets over it. Engineering economics and delayin disconnection and connection operations are suitably considered inselecting the type of embodiment to use there.

When operations are in state (s11,d11), e.g. (16.0, 0.0) and either agatekeeper request GK=1 or Buffer Occupancy BFR=1 occurs, then adaptivesource rate measures are employed without diversity measures, in thisexample. In such case, a transition 2215 goes from (s11,d11) to state(s31,d31), e.g., (8.0,0.0) kbps of source rate and no diversity rate.After transition 2215, operations remain at state (s31,d31) unless anduntil both the gatekeeper request is turned off and the Buffer Occupancycondition is not present, i.e. GK=0 AND BFR=0. At that point, statetransition 2231 takes the system from state (s31,d31) to state (s21,d21)e.g., (11.2,0.0). By transition 2231, source rate is increased, anddiversity remains off. Operations at state (s21,d21) poll the gatekeeperand buffer for updated status information. Then operations remain atstate (s21,d21) unless the GK and BFR remain off, or in case GK or BFRgo on again and later become both off, i.e. GK=0 AND BFR=0. Thereupon astate transition 2233 transfers the system to state (s11,d11) e.g.,(16.0,0.0). By transition 2233, source rate is increased, and diversityrate remains disabled or at zero, and their sum (overall transmissionrate) is increased. Operations remain at state (s11,d11) unlesscriterion F causes aggressive transition 2211 of FIG. 23 or moderatetransition 101 of FIG. 1 or unless GK or BFR go on again to causetransition 2215 of FIG. 23.

The processes and systems of FIGS. 1 and 23 importantly introduce newcriteria for transition, as in steps 101 combined with 2211 forinstance, and combine the new criteria for transition with discretestates (sij,dij). Advantageously, the use of discrete states with thesenew criteria reduces the incidence of false alarms and oscillations.

Note that the state transition diagrams of FIGS. 1, 22 and 23 compactlyshow advantageous features of some embodiments. Further note that theflowcharts of FIGS. 16, 24, 25 and 26 show some of the same things asthe state transition diagrams and also further advantageous features ofprocesses, devices and systems as taught herein.

FIG. 24 shows a process for implementation in software media, integratedcircuits, printed circuit cards, personal computers, networkedappliances and other network edge-device computers, cell phone basestations, servers, routers, gateways and other apparatus. This processsupports conferencing and multicasting.

The adaptive rate/diversity improvements are suitably implemented inconferencing, broadcast, unicast, and multicast devices and processessince UDP universal datagram protocol, RTP real-time transport protocoland RTCP and other protocols now available or yet to be devised areuseful for supporting these services.

Broadcast with path diversity is described in connection withincorporated patent U.S. Pat. No. 6,496,477 FIG. 11. Conventionalbroadcast replicates the process of a single unicast connection fromsource to destination so that communication of a media stream isdirected to many destinations. Improving upon conventional broadcast,adaptive rate/diversity processes as taught herein are replicated sothat the media stream takes diverse packets to each of manydestinations, and adaptive control of rate and time or path or combinedtime/path diversity as taught herein is applied to the communicationseach by each.

Multicast with path diversity is described in connection withincorporated patent U.S. Pat. No. 6,496,477 FIG. 12. Conventionalmulticasting fans out a media stream from a source farther out in thepacket network so that communication of a media stream is directed tomany destinations. Improving upon conventional multicast, adaptiverate/diversity process as taught herein is applied to thecommunications. The situation differs from adaptive rate/diversitycontrol of improved broadcasting as described in the previous paragraphbecause rate/diversity adaptation of a given media stream at the source1111 of U.S. Pat. No. 6,496,477 FIG. 12 affects plural destinations.When the plural destinations are experiencing different levels of QoS asreported in their RTCP packets sent back to source 1111, then therate/diversity adaptation thus can be faced with conflicting QoSinformation to reconcile in making an adaptation transition such as 101of FIG. 1.

Before proceeding further, note an example process context in FIG. 24.Operations commence at a BEGIN 2401 and proceed to establish an initialstate (s11,d11) in a step 2411. Next a step 2421 inputs RTCP reportpackets, but now from multiple destinations. A step 2431 generatesinformation called herein a “report datum” for each destination, in someembodiments. Next a step 2441 generates a value from the “report data”collectively. This special value is called MQoS, motivated by but notlimited to a concept of a Multicast QoS. Then the special MQoS value isused in place of Loss Fraction L in a step 2451 to drive the adaptiverate/diversity process steps at the sender which are collectively calledprocess 2461. Process 2461 includes the steps 1613 through 1671 of FIG.16 used in the particular part of FIG. 24 identified by numeral 2461.The process 2461 selectively and adaptively updates the NEWSTATE whenappropriate and operations loops back to step 2421 to input more RTCPreport packets as the process goes forward in time. When process 2400 isto be turned off, operations go from STOP decision step 1671 to RETURN2471.

The operations of steps 2441 and 2431 are next described in considerabledetail. Note that there are many alternative ways of doing each of them,and an outline format is used to facilitate the detailed description.

Accordingly, several embodiments of adaptive rate/diversity control ofimproved multicasting are contemplated. As noted hereinabove,rate/diversity adaptation of a given media stream at the source 1111 ofU.S. Pat. No. 6,496,477 FIG. 12 affects plural destinations. QoS reportscome back to source 1111 from the various destinations for each portionin a series of portions comprising the transmission from source 1111.The QoS reports from the various destinations for a given portion of thetransmission are combined into one or more herein-defined “MulticastQoS” evaluation numbers in FIG. 24 step 2441 to drive the rate/diversityadaptation processes. Multicast QoS (MQoS) is variously defined fordifferent process and device embodiments next. (The value of S, meaningestimated steady state overall transmission rate as in step 1631, isalso chosen in a similar manner to compute what is herein called“Multicast S”. In other words, use the information from each destinationto compute an S value for that destination. Then, in a manner preciselyanalogous to any selected one of the MQoS calculations below, computethe Multicast S from the S values).

A. In a first method, the QoS reports from the various destinations fora given portion of the transmission are combined into a singleherein-defined “Multicast QoS” evaluation number in step 2441 to drivethe rate/diversity adaptation processes. In other words, Multicast QoS(MQoS) is defined for each corresponding transmission portion such asactivity in a 5-second interval described by an RTCP report packet. Thestep 2431 of FIG. 24 simply uses the Loss Fraction datum in one RTCPreport packet as a report datum (or computes criterion F from data likeLoss Fraction and Delay Jitter in one RTCP report packet) or computescriterion F using QoS computation methods described with reference toFIGS. 1 and 23 for instance.

A1. The MQoS in step 2441 depends on what happens to fewer than all ofthe destinations

A1a. The MQoS depends on what happens to a majority of the destinations.Example: For the 37^(th) RTCP packet, QoS values came back from 150 outof 155 destinations. The A1a embodiment is programmed to find the valueof a statistic based on the QoS values from the best-QoS reporting X %of the 155 destinations, e.g., (say 90% of them), the best140(=155×0.90) out of the 150 QoS values.

-   -   A1ai. MQoS=average of the best reporting X %    -   A1aii. MQoS=minimum of the best reporting X %    -   A1aiii. MQoS=median of the best reporting X %    -   A1aiv. MQoS=average of the worst reporting Y %    -   A1av. MQoS=minimum of the worst reporting Y %    -   A1avi. MQoS=median of the worst reporting Y %    -   A1avii. MQoS=maximum of the worst reporting Y %.

A1b. The MQoS depends on what happens to selected ones of thedestinations. Example: For the 23rd RTCP packet, QoS values came backfrom 205 out of 324 destinations. The A1b embodiment is programmed tofind the value of a statistic based on the QoS values disregarding thebest-QoS reporting X % of the 324 destinations and disregarding theworst-QoS reporting Y % of the 324 destinations. MQoS=average of 20percentile to 90 percentile loss fraction RTCP reports.

A1c. The MQoS depends on randomly selected ones of the destinations.Example: For the 147th RTCP packet, QoS values came back from 2000 outof 3000 destinations. The A1b embodiment is programmed to find the valueof a statistic based on a random sample of N=100 of the 2000 QoS values.Then the processes apply a statistical computation according to any ofthe following alternatives:

-   -   A1ci. MQoS=average of the N selected QoS values    -   A1cii. MQoS=minimum of the N selected QoS values    -   A1ciii. MQoS=median of the N selected QoS values    -   A1civ. MQoS=average of the selected QoS values, but        -   leaving out their top X % and bottom Y %.

A2. A rate/diversity adaptation decision step depends on a MQoSstatistic based on all the destinations' reports of QoS received backfor the given transmission portion.

A2a. The statistic is the median QoS. Example: For the 7^(th) RTCPpacket, QoS values came back from 150 destinations out of 180destinations. The loss fraction values varied from 0.2% to 12%, mostlyaround 3%. 2.8% loss fraction was the median value. MQoS=2.8%.

A2b. The statistic is the QoS of the destination at the nth percentileof QoS. Example: For the 17^(th) RTCP packet, QoS values came back from138 destinations out of 180 destinations. The embodiment is programmedto find the 30^(th) percentile as indicated by listing the reports inloss fraction order. Example: The loss fraction values varied from 0.2%to 12%, mostly around 3%. The 30^(th) percentile value was 6.2%.MQoS=6.2%.

A2c. The statistic is the average QoS. Example: For the 24^(th) RTCPpacket, QoS values came back from 155 destinations. The embodiment isprogrammed to find the arithmetic mean or average QoS. Example: The lossfraction values varied from 0.2% to 12%, mostly around 3%. Thearithmetic mean was 3.3%. MQoS=3.3%.

A2d. The statistic is the minimum QoS of any destination. Example: Forthe 33rd RTCP packet, QoS values came back from 125 destinations. Theloss fraction values varied from 0.2% to 12.2%, mostly around 3%. Theminimum QoS was 12.2% loss fraction. MQoS=12.2%.

In a second process type, the QoS reports from the various destinationsfor two or more portions of the transmission are combined into a singleherein-defined “Multicast QoS” evaluation number to drive therate/diversity adaptation processes. Two or more RTCP packets from thesame destination are used to generate each report datum by averaging,median, minimum or other statistic, and step 2431 becomes more detailed.If only one RTCP packet from a given destination comes back when a mostdestinations are reporting back three for the given process, then thereport datum is the value of that RTCP packet. The report data thusderived destination by destination are used according to any of theA-numbered processes to generate MQoS by step 2441 of FIG. 24, accordingto a correspondingly B-numbered embodiment.

Example Embodiment B2a: The statistic is the median QoS. Example: Forthe 7^(th) through 12th RTCP packets, QoS values came back from 150destinations. Each set of six RTCP packet Loss Fraction values for agiven destination was averaged to produce a report datum for thatdestination. The report data varied from 0.5% to 7%, mostly around 3%.2.9% loss fraction was the median value. MQoS=2.9%.

A tedious description of other B-type embodiments is suitably generatedby following the directions of the previous paragraph, which is believedto amply disclose the subject matter of numerous B-type embodiments.

Other QoS level measures and adaptation logics are suitably combinedwith the teachings and figures shown herein.

FIG. 25 illustrates software to implement each of steps 1621, 1623 and1629 of FIG. 16. In FIG. 25, after a BEGIN, operations go to a step 2511to determine whether a diversity flag is on. If not, then source ratesij is decreased in a step 2515. Next after step 2515, a step 2521 sets,or turns on, the diversity flag. Then a step 2531 calls a diversityroutine to create diversity packets. Then a step 2541 updates packetheader diversity fields and dependency information appropriately. Thosefeatures of the packet are described in U.S. Pat. No. 6,496,477 for pathdiversity. Then a RETURN 2571 is reached.

If in step 2511 the diversity flag is already on, then operations branchto a step 2551 to vary the source rate and diversity aspects of thecoder. Then in step 2561, the packet header is updated in the dependencyinformation and diversity fields to correspond to the changes made bystep 2551, whereupon RETURN 2571 is reached.

In FIG. 26, step 1631 of FIG. 16 commences and goes to an input step2605. In step 2605, information specifying a steady state overalltransmission rate S is either estimated locally or input from a networkelement. Next, operations go to a decision step 2611 to determinewhether the diversity flag is on. If so, operations go to a decisionstep 2621 to determine whether the overall transmission rate sij+dij isequal to estimated steady state overall transmission rate S from step2605, e.g., 11.2. If an estimated steady state overall transmission rateS value is not available, then a maximum amount (e.g., 16.0) is used forS by default. If so, then operations turn off (reset) the diversity flagin a step 2631. Next a step 2641 calls the diversity routine to reducediversity. Then a step 2651 updates packet header diversity fields anddependency information appropriately. Those features of the packet aredescribed in U.S. Pat. No. 6,496,477 for path diversity, and the pathdiversity routine is described in FIG. 18 therein. Then a RETURN 2681 isreached.

If in step 2621, the overall transmission rate is below value S, thenoperations branch to a step 2661 to change both the source rate anddiversity (and suitably the path diversity method of the PacketTransmission Table of U.S. Pat. No. 6,496,477 and other diversitymethods) without closing down the diversity feature. Then operations goto step 2651 to update packet header as described above.

If in step 2611, the diversity flag is off, then operations branch to astep 2671 to increase the source rate only, whereupon RETURN 2681 isultimately reached.

FIG. 27 shows how adaptive multipath routing is combined with adaptiverate/diversity to form a new combination process for integratedcircuits, and systems of all kinds. In one embodiment, diverse paths viathree particular proxies are identified as described in incorporatedU.S. Pat. No. 6,496,477, FIGS. 18-25. A first path via the first proxyis maintained throughout a communications connection. A second path isestablished via the second proxy, but when packet loss becomesunacceptable the second path is reestablished via the third proxy. Inanother more complex embodiment of FIG. 27, the multipath routingprocess seeks a satisfactory path and may switch adaptively from onepath to another. Concurrently, the multipath routing process has itssource rate adaptively varied. Also concurrently, the multipath routingprocess has one or more additional adaptive multipath routing process“siblings” seeking a respective second satisfactory path for diversitypackets and switching adaptively from one second path to another.Advantageously, a path diversity receiving process implemented in thedestination operates as shown and described in connection with FIGS. 5,17 and 26 of incorporated patent U.S. Pat. No. 6,496,477 and/or aselsewhere described therein. In this way, complex and hard-to-solvenetwork congestion problems are addressed by improved embodiments asillustrated by FIG. 27.

FIG. 28 supplements the software blocks of FIG. 18 by adding ATM(asynchronous transfer mode), AAL (ATM Adaptation Layer) and Frame RelaySoftware coupled to the IP block in a TCP/UDP/IP software stack.

In FIG. 29, a local software application of rate/diversity controlswitches between states within the same oval (same overall transmissionrate). Thus, diversity allocation is done by the DSP softwareapplication.

FIG. 29 also illustrates the concept that not just two, but three oreven more states per rate oval are suitably introduced. One state peroval has source rate only, with no diversity. Another state per oval haspackets with a source rate and a diversity packet with its diversityrate. A third state per oval has packets with a source rate, plus twodiversity packets with respective diversity rates. Various examples of athird state are shown in FIG. 6, diversity packets 621, 631 and 641.Furthermore, path diversity alternatives are illustrated in theincorporated patent U.S. Pat. No. 6,496,477 such as in the PacketTransmission Table therein.

In a complementary way, the network advantageously controls overalltransmission rate, so that transitions between ovals in FIG. 29 areunder network control, such as by a gatekeeper.

In FIG. 29, operations suitably begin at state (16,0) at left. Anestimation EST of network congestion is computed by network or by senderor by receiver to determine whether to make a high priority source rateadjustment. If operations are in any of the state of the 16 kbps ovaland EST=8, then a transition goes from the originating 16 kbps state tostate (4.0,1.7,2.3) on far right. If EST is neither of 11.2 or 8 then noEST driven transition is executed. If EST=11.2 as signaled by thenetwork (or alternatively estimated by sender or receiver), then atransition goes from the originating 16 kbps state to state(5.7,2.3,3.2).

If at any 11.2 kbps state estimation EST=8, then a transition goes fromthe originating 11.2 kbps state to state (4.0,1.7,2.3). If GK=0 ANDBFR=0 is signaled by the network, then a transition goes from anyoriginating 8 kbps state back to state (5.7,2.3,3.2) provided that aratio R is also less than or equal to a threshold Th3. For example, R isthe ratio of estimated steady state overall transmission rate divided bycurrent overall transmission rate. Th3 suitably lies in a range of 1.5to 4.0, and a value of 3.0 is suitable. Estimated steady state overalltransmission rate S is that rate which the network signals is nowavailable or which test algorithms at sender or receiver indicate is nowavailable. Rate S is suitably computed as in the discussion of FIG. 16step 1631.

If at any 11.2 kbps state, GK=0 AND BFR=0 is signaled by the network,then a transition goes from the originating 11.2 kbps state to state(8,3.2,4.8). If at any 8 kbps state, network conditions indicate greatlylessened congestion, then an aggressive recovery to state (8,3.2,4.8) isdesirable. In FIG. 29, criterion (R>Th3) AND (GK=BFR=0) triggers atransition from any originating 8 kbps state to state (8,3.2,4.8) whenthe criterion is met.

Within a given oval, DSP software determines from QoS reports whether tomake transitions to add diversity, relax diversity, or releasediversity. Starting at state (16,0), a determination that A≧F>Th1 addsdiversity and takes operations to a state (11.2,4.8). However, startingat state (16,0), if F>A>Th1, then operations adds two stages ofdiversity and goes to a state (8.0, 3.2,4.8). Starting at state(11.2,4.8), a determination that F>Th1 adds further diversity and takesoperations to state (8.0,3.2,4.8). If QoS becomes ameliorated, such thatF<Th2, then operations are transitioned from state (8.0, 3.2,4.8) tostate (11.2,4.8) and/or from state (11.2,4.8) to state (16,0) as shown.

Starting at state (11.2,0), a determination that A≧F>Th1 adds diversityand takes operations to a state (8.0,3.2). However, if F>A>Th1, thenstarting at (11.2,0) operations become more aggressive and add twostages of diversity and go to a state (5.7,2.3,3.2). If F>Th1 continues(unacceptable QoS) at state (8.0,3.2), then operations add diversity andgo from state (8.0,3.2) to the state (5.7,2.3,3.2). If QoS becomesameliorated, such that F<Th2, then operations are transitioned fromstate (5.7, 2.3,3.2) to state (8.0,3.2) and/or from state (8.0,3.2) tostate (11.2,0) as shown.

Starting at state (8.0,0), a determination that A≧F>Th1 adds diversityand takes operations to a state (5.7,2.3). However, if F>A>Th1, thenstarting at (8.0,0), operations become more aggressive and add twostages of diversity and go to a state (4.0,1.7,2.3). If F>Th1 continues(unacceptable QoS) at state (5.7,2.3), then operations add diversity andgo from state (5.7,2.3) to a state (4.0,1.7,2.3). If QoS becomesameliorated, such that F<Th2, then operations are transitioned fromstate (4.0,1.7,2.3) to state (5.7,2.3) and/or from state (5.7,2.3) tostate (8.0,0) as shown.

With the use of ratio R in the transition criteria, overall transmissionrate changes from low rate ovals are advantageously arranged to belarger than rate changes on return transitions between higher rateovals. Thus, successively smaller increases in rate are achieved withfiner increases as higher rates (and attendant network burden) areapproached.

FIG. 30 shows a histogram of frequency in percent versus number # ofconsecutive packet losses in a window time interval such as 5 seconds.The instances of packet losses are tabulated zero (no loss) where apacket is received, one (1: only one packet lost and not two or moreconsecutively), two (2: two packets lost consecutively and not 3 or moreconsecutively), three (3: three packets lost consecutively and not 4 ormore consecutively) and four plus (4+: four or more packets lostconsecutively).

The histogram information is here recognized as quite useful forrate/diversity adaptation purposes. Even though the packet loss ratemight be the same in two different cases, the aggressiveness ofadaptation measures is suitably made more aggressive if the histogram ismore populated with higher numbers of packets lost consecutively.

FIG. 31 shows a state transition diagram for implementing selectivelyaggressive measures based on the histogram information of FIG. 30. Thereceiver 361′ of FIG. 3 collects the information of the histogram andacts upon it directly, or sends the information of the histogram back tosender 331 for rate/diversity adaptation in sender 331.

In FIG. 31 operations suitably are arranged to begin at a high sourcerate and low (or zero) diversity state (s11,d11). Different criteriacalled z1 and z2 cause respectively moderate and aggressive adaptationmeasures based on the consecutive packet loss histogram information. Ifz1 occurs, then operations go from state (s11,d11) to reduce source rateand introduce an amount d22 of single diversity at state (s22,d22). Thisis the moderate adaptation.

If z2 occurs, then operations instead go from state (s11,d11) to reducesource rate and introduce two amounts of diversity d42,e42 at state(s42,d42,e42). This is the aggressive adaptation.

Criterion z1 is suitably established as (A≧F>Th1) AND frequency oftwo-consecutive-losses- or more is Th2 or less.

Criterion z2 is suitably established as (F>A>Th1) OR frequency oftwo-consecutive-losses- or more exceeds Th2.

If and when QoS becomes ameliorated, such that a criterion z3 is met,then operations are transitioned from state (s42,d42,e42) to state(s22,d22), and/or from state (s22,d22) to state (s12,d12), and/or fromstate (s12,d12) to state (s11,d11) as shown.

Criterion z3 is suitably established as (F<Th3) AND frequency oftwo-consecutive-losses- or more is less than a threshold Th4.

Th1 is suitably made 3%, Th2 is suitably 2%, Th3 is suitably 0.5% andTh4 is suitably 0.25%. The skilled worker suitably tunes the thresholds.Also, an automated tuning process suitably varies the thresholds overillustrative ranges 1-5% for Th1 and Th2, and over 0% to 2% for Th3 andTh4 for most satisfactory adaptation operation.

Note among other advantageous features of the process of FIG. 31 thatthe return transition from state (s42,d42,e42) to state (s22,d22) issuitably made to be a larger transition in overall transmission ratethan the subsequent return transition from (s22,d22) to (s12,d12) andthence to (s11,d11). In this way, a smoother servo homing behavior isachieved.

FIG. 31 is interpreted in light of the embodiments and transitioncriteria earlier discussed herein, and it should be apparent thatnumerous embodiments varying the arrangements illustrated in FIG. 31 arealso contemplated based on mixing and matching various criteria fromother embodiments.

FIG. 32 illustrates process, device and system embodiments applying afurther concept of varying the number of frames per packet (frm/pkt) bytransition from a state 3211 (1 frm/pkt) to a state 3221 (2 frm/pkt)when A≧F>Th1. When F>A>Th1 in state 3211, a more aggressive adaptationtransition to a state 3231 (3 frm/pkt) occurs.

Bracketed sets of packets illustrate the meaning of each state. State3211 corresponds to transmission of packets in a series of packets eachwith a header H and a payload comprising one frame of compressed data,sent at a certain number of packets per second and a certain number offrames per second.

A second state 3221 involves transmission of packets in a series ofpackets each with a header H and a payload comprising two frames ofcompressed data sent suitably (but not necessarily) at the same numberof frames per second as in state 3211, but at a different and fewernumber of packets per second. Notice that a brace indicates 3 payloadframes corresponding to comparable information distributed differentlyamong packets depending on which state is used.

A third state 3231 involves transmission of packets in a series ofpackets each with a header H and a payload comprising three frames ofcompressed data sent suitably (but not necessarily) at the same numberof frames per second as in state 3211, but at a different and stillfewer number of packets per second. Notice that 3 payload frames areincluded in the same one packet with its one header when state 3231 isused.

Return transitions occur when criterion F<Th2. One return transitiontakes operations from state 3231 to state 3221. Another returntransition takes operations from state 3221 to state 3211.

Note that the criteria for transition are suitably selected according toany of the various embodiments elsewhere described herein.

The variable frames-per-packet embodiments are suitably augmented withtime or path or combined time/path diversity as shown in FIG. 33. Notethat ovals surround states that have the same overall transmission ratesij+dij. Suppose a transmission rate of header bits is 8 kbps ofoverhead (ovhd), for example, when the source rate is 16.0 kbps with oneframe per packet. Then in other ovals the transmission rate overhead ofheader bits is the 8 kbps base rate divided by the number of frames perpacket. So ovhd=4 kbps at 2 frm/pkt, and ovhd=8/3 kbps at 3 frm/pkt.

FIG. 33 illustrates process, device and system embodiments applyingdiversity and variable number of frames per packet (frm/pkt) usingmultiple description (MD) technology. In FIG. 33 operations transitionfrom a state 3211 (1 frm/pkt, 16 kbps source rate, zero diversity rate)to a state 3325 (2 frm/pkt, (5.6 kbps source rate, 5.6 kbps diversityrate)) when A≧F>Th1. When F>A>Th1 in state 3211, a more aggressiveadaptation transition to a state 3335 (3 frm/pkt, (4 kbps source rate, 4kbps diversity rate)) occurs. Further in FIG. 33, a network conditionGK=10R BFR=1 occurring when operations occupy state 3211, transitionsthe operations to a state 3221 (2 frm/pkt, 11.2 kbps, zero diversity).

Return transitions occur when criterion F<Th2. One return transitiontakes operations from state 3335 to state 3325. Another returntransition takes operations from state 3325 to a state 3315 (1 frm/pkt,8 kbps source rate, 8 kbps diversity rate). Another return transitiontakes operations from state 3315 to state 3211.

Flow diagrams of some processes for control of frames/packet are thesame as FIG. 16 except the update-NEWSTATE steps 1621, 1623, 1629 and1631 are programmed for frames per packet control. Thus, FIG. 25 steps2515 and 2551 are enhanced by incrementing frames/packet therein. Also,FIG. 26 steps 2661 and 2671 are enhanced by decrementing frames/packetcontrol bits to control RTP packet encapsulation 341 of FIG. 3 andpacket encapsulation unit 1571 of FIG. 15. Packet headers are suitablyupdated with new frames/packet information as desired in steps 2541,2561 and 2651.

Again, other criteria for the transitions as described elsewhere hereinare suitably employed. Each of the types of time diversity, pathdiversity, and time/path diversity as described herein and in theincorporated U.S. Pat. No. 6,496,477 are contemplated for use in variousembodiments.

Note that changing number of frames per packet, it may be advisable insome embodiments to make the frm/pkt transition only during a silenceperiod following a talkspurt featuring unacceptable QoS. Otherembodiments suitably make frm/pkt transition during a talkspurt withoutrestriction.

Another embodiment performs a hybrid frame/packet adaptation:frame/packet increase occurs during both silence periods and activespeech, but frame/packet decrease occurs during silence periods only.Steps 2515, 2551, 2661 and 2671 are correspondingly improved bypreceding them with tests for presence of a talkspurt flag or a silenceflag, so that the transitions occur according to the just mentionedlogic embodiments that depend on silence only, or talkspurt, or duringeither silence or talkspurt. If the required test is not met, therespective step 2515, 2551, 2661 or 2671 is bypassed, and if the test ismet the respective said step is performed.

Also, note a possible effect on some diversity methods when changingnumber of frames per packet. Suppose, for example, that a time diversityP(n)P(n−1)′ in one packet is changed to P(n)P(n−1)′ P(n+1)P(n)′ bychanging to more frames per packet. If the longer packet is lost, bothP(n)′ and P(n) are lost, meaning that all of the nth information islost. Accordingly, some embodiments suitably change to a diversitymethod that is resistant to packet loss concurrently with (or at leastclose in time to) a transition from one number of frames per packet to ahigher number of frames per packet.

Gateways, wireless base stations, private branch exchanges, networkedappliances and other applications are suitably enabled by adaptiverate/diversity controls, chips, chipsets, printed circuit cards, andsubsystems disclosed herein. Recoder and/or transcoding processesrecodes or transcode the information and produces an output compressedand coded according to a different form than was received by a givendevice. It is contemplated that devices, processes and systems aresuitably cascaded and integrated for various telecommunication andnetworking purposes. Where many channels are processed simultaneously,the systems are suitably replicated or multiplexed to the extentdesired, so that software and hardware are effectively, efficiently andeconomically employed. Where blocks are shown herein, they are suitablyimplemented in hardware, firmware or software in any combination. Theembodiments described are merely illustrative, while the scope of theinventive subject matter is defined by the claims and equivalentsthereof.

What is claimed is:
 1. A process of sending packets on a network,comprising: A. sending source packets of real-time, voice informationfrom a sender to a receiver on the network at a first source rate thatis equal to a first transmission rate; B. receiving at the sender packetloss information based on a loss of source packets at the receiver; C.comparing in the sender the packet loss information with a thresholdnumber; and D. when the packet loss information is greater than thethreshold number, sending the source packets at a second source rate andsending at least one of time and path diversity of the real-time, voiceinformation at a first diversity rate, the sum of the second source rateand the first diversity rate being equal to the first transmission rate.2. The process of claim 1 including comparing in the sender the packetloss information with a threshold number of three percent.
 3. Theprocess of claim 1 in which the sending at least one of time and pathdiversity includes sending diversity packets with the source packets. 4.The process of claim 1 in which the receiving includes receiving a RealTime Transport Control Protocol report packet.
 5. The process of claim 1including sending the source packets at the first source rate of 16.0kilobits per second and the sum of the second source rate and the firstdiversity rate is 11.2 kilobits per second.
 6. The process of claim 1including sending the source packets at the first source rate of 11.2kilobits per second and the sum of the second source rate and the firstdiversity rate is 8.0 kilobits per second.
 7. The process of claim 1including sending the source packets at the first source rate of 8.0kilobits per second and the sum of the second source rate and the firstdiversity rate is 5.7 kilobits per second.
 8. The process of claim 1 inwhich the sending the source packets at a second source rate includescoding the voice information with more compression than when sending thesource packets at the first source rate.